ORM

Django's object-relational mapper is the largest subsystem in the codebase: roughly 30,000 lines under django/db/, plus another ~15,000 lines per database backend. It exposes a model-centric API on top of a query builder that ultimately produces parameterised SQL for SQLite, PostgreSQL, MySQL/MariaDB, or Oracle.

Purpose

Provide a Pythonic API for defining database tables (Model), querying them (QuerySet), and managing schema changes (covered separately under migrations). The ORM is opinionated: every model has a single primary key, every query is composable and lazy, and joining is implicit through ForeignKey and ManyToManyField declarations.

Directory layout

django/db/
├── __init__.py              # exposes `connection`, `connections`, `transaction`
├── transaction.py           # atomic(), commit/rollback hooks, savepoints
├── utils.py                 # ConnectionHandler, error wrapping, OperationalError, IntegrityError
├── backends/
│   ├── base/                # backend base classes (DatabaseWrapper, SchemaEditor, Operations)
│   ├── sqlite3/             # SQLite implementation
│   ├── postgresql/          # PostgreSQL implementation
│   ├── mysql/               # MySQL/MariaDB implementation
│   ├── oracle/              # Oracle implementation
│   ├── dummy/               # No-op backend for tests
│   └── ddl_references.py    # DDL fragment helpers (Table, Column, ForeignKey, …)
├── migrations/              # see systems/migrations.md
└── models/
    ├── base.py              # Model + ModelBase metaclass
    ├── manager.py           # Manager
    ├── query.py             # QuerySet, RawQuerySet, prefetch_related
    ├── query_utils.py       # Q, FilteredRelation
    ├── expressions.py       # F, Func, Value, Window, Case/When, Subquery
    ├── lookups.py           # exact, gte, icontains, in, …
    ├── aggregates.py        # Count, Sum, Avg, Min, Max, StdDev, Variance
    ├── functions/           # database functions (Coalesce, Concat, Cast, Now, Trunc, …)
    ├── fields/              # field classes (CharField, IntegerField, ForeignKey, …)
    ├── deletion.py          # cascade rules, on_delete handlers
    ├── constraints.py       # CheckConstraint, UniqueConstraint, ExclusionConstraint
    ├── indexes.py           # Index
    ├── options.py           # Meta options resolution
    ├── signals.py           # pre_save, post_save, pre_delete, post_delete
    └── sql/
        ├── query.py         # the internal Query object
        ├── compiler.py      # SQL generation
        ├── where.py         # WhereNode (boolean tree of constraints)
        └── subqueries.py    # UpdateQuery, DeleteQuery, AggregateQuery

Key abstractions

Type	File	Role
Model	django/db/models/base.py	Base class users subclass; metaclass wires fields to columns
Manager	django/db/models/manager.py	Returns querysets; Model.objects is one
QuerySet	django/db/models/query.py	Lazy, chainable query builder
Q	django/db/models/query_utils.py	Composable boolean expression
F	django/db/models/expressions.py	Column reference (database-side)
Field	django/db/models/fields/__init__.py	Column descriptor (subclassed for each type)
Lookup	django/db/models/lookups.py	Filter operator (exact, gte, etc.)
Query	django/db/models/sql/query.py	Internal query representation
SQLCompiler	django/db/models/sql/compiler.py	Turns Query into SQL + params
BaseDatabaseWrapper	django/db/backends/base/base.py	Per-connection driver wrapper
BaseDatabaseSchemaEditor	django/db/backends/base/schema.py	DDL emission
BaseDatabaseOperations	django/db/backends/base/operations.py	Backend-specific SQL fragments
BaseDatabaseFeatures	django/db/backends/base/features.py	Feature flags

How it works

graph TD
    User["model.objects.filter(name='x').order_by('-id')"]
    Manager
    QuerySet
    Query["sql.Query<br/>where: WhereNode<br/>group_by, order_by, joins"]
    Compiler["SQLCompiler<br/>as_sql() -> (sql, params)"]
    Wrapper["DatabaseWrapper<br/>cursor.execute(sql, params)"]
    DB[(Database)]
    Rows
    Iterable

    User --> Manager
    Manager --> QuerySet
    QuerySet --> Query
    Query -->|get_compiler| Compiler
    Compiler --> Wrapper
    Wrapper --> DB
    DB --> Rows
    Rows --> Compiler
    Compiler -->|model instances| QuerySet
    QuerySet --> Iterable

Querysets are lazy

A QuerySet is a description of a query, not the result of one. None of filter, order_by, annotate, or select_related execute SQL. The query runs on:

• Iteration (for obj in qs).
• len(qs) / bool(qs).
• Slicing with a step (qs[1::2]).
• Pickling.
• Methods that explicitly evaluate (get, count, exists, first, last, aggregate, update, delete, …).

Internally, QuerySet._fetch_all() (in django/db/models/query.py) is the entry point that triggers compilation. It populates _result_cache so subsequent iterations are free.

The Query object is mutable; the QuerySet is not

User-facing methods on QuerySet are all "clone and mutate the clone". QuerySet._chain() deep-clones the underlying Query object, applies the change, and returns a fresh QuerySet. This is why qs.filter(...) returns a new queryset rather than mutating in place.

Compilation is a tree walk

SQLCompiler.as_sql() walks the Query's tree of WhereNode (boolean tree), Expression (columns, functions, subqueries), and BaseTable/Join nodes (FROM clause), emitting SQL fragments. Each backend's Operations class overrides specific fragments (quote_name, format_for_duration_arithmetic, convert_durationfield_value, …).

Backends override surgically

BaseDatabaseWrapper provides the base cursor(), _commit(), _rollback(), savepoint, transaction, and connection methods. Per-backend wrappers add only what's necessary:

• sqlite3/base.py adds in-memory database handling and the _sqlite_* registered functions for Trunc, etc.
• postgresql/base.py adds psycopg-specific cursor handling, pg_trgm, range types.
• mysql/base.py adds connection options, JSON path translation, MySQL-specific quoting.
• oracle/base.py adds the heaviest customisations because Oracle differs the most.

The schema editor (<backend>/schema.py) similarly overrides only the DDL fragments that differ.

Transactions

django/db/transaction.py provides:

• atomic() — context manager and decorator. Wraps a block in a transaction; nested atomic uses savepoints.
• commit() / rollback() — manual control (autocommit must be disabled).
• on_commit(callback) — register a callback that fires only if the outer transaction commits successfully.
• set_autocommit(False) / set_autocommit(True) — toggle autocommit mode.

The atomic block tracks state on connection.savepoint_ids and connection.in_atomic_block. Failures propagate up; each level decides whether to release or roll back the savepoint.

Async ORM

Most QuerySet methods have async siblings prefixed with a:

• aget, acount, aexists, afirst, alast
• acreate, aupdate, adelete, asave
• aiterator

These wrap the synchronous implementation with asgiref.sync.sync_to_async. The actual database I/O is still synchronous — the database driver doesn't run asynchronously — but it's pushed off the event loop. PostgreSQL's psycopg3 async support is being explored but is not yet wired in by default.

Integration points

• Migrations (migrations) read model state via the apps registry and emit DDL through the schema editor.
• Forms (forms) generate ModelForm from model fields via forms/models.py.
• Admin (contrib) builds list views, change views, and filters from ModelAdmin declarations and the underlying ORM.
• Signals (models/signals.py) fire on save/delete; contenttypes and auth listen.
• System checks in django/core/checks/model_checks.py and per-field check() methods enforce model validity at startup.
• Caching (django/utils/cache.py) uses ORM-derived cache keys for cache_page.

Entry points for modification

• New field type: subclass Field in django/db/models/fields/__init__.py. Override db_type(), to_python(), from_db_value(), get_prep_value(). Register lookups via Field.register_lookup(). See JSONField in fields/json.py for a complete example.
• New lookup: subclass Lookup or Transform in django/db/models/lookups.py. Register on the field class with Field.register_lookup.
• New aggregate or function: add to django/db/models/aggregates.py or django/db/models/functions/. Implement as_sql() for cross-backend behaviour and as_<vendor>() for backend-specific tweaks.
• Backend feature: edit django/db/backends/<engine>/features.py. Tests in tests/backends/ and the per-feature test apps will exercise the change.
• Schema editor change: django/db/backends/base/schema.py defines the cross-backend interface; per-backend schema.py files override.

Key source files

File	Purpose
django/db/models/base.py	Model and ModelBase metaclass; save/delete logic
django/db/models/query.py	QuerySet, prefetch machinery, async dispatch
django/db/models/sql/query.py	The internal Query object
django/db/models/sql/compiler.py	SQL generation
django/db/models/expressions.py	F, Func, Value, Window, Case/When, Subquery
django/db/models/fields/__init__.py	Built-in field classes
django/db/models/fields/related.py	ForeignKey, OneToOneField, ManyToManyField
django/db/models/manager.py	Manager and the from_queryset builder
django/db/models/options.py	Meta options (db_table, unique_together, …)
django/db/models/deletion.py	Cascade resolution
django/db/backends/base/base.py	BaseDatabaseWrapper
django/db/backends/base/schema.py	BaseDatabaseSchemaEditor
django/db/transaction.py	atomic() and friends
django/db/utils.py	Connection routing, error wrapping

Where to read tests

• tests/queries/ — exhaustive QuerySet coverage.
• tests/lookup/ — field lookups.
• tests/expressions/ — F, Func, Subquery, Window.
• tests/aggregation/ — aggregates with grouping.
• tests/many_to_many/, tests/many_to_one/, tests/one_to_one/ — relation behaviour.
• tests/transactions/ — atomic semantics.
• tests/backends/ — per-backend feature checks.
• tests/schema/ — schema editor tests.