Database

GitLab runs against PostgreSQL 16+ with up to four logical databases, dynamic partitioning, custom load balancing, and ~1,500 migrations.

Purpose

Persist almost every piece of GitLab state — users, projects, MRs, pipelines, security findings, etc. The schema is the single largest design artifact in the repo.

The four databases

Database	What's in it	Models	Migration namespace
main	Users, projects, groups, issues, MRs, notes	ApplicationRecord	db/migrate/
ci	Pipelines, builds, jobs, runners, artifacts	Ci::ApplicationRecord	shared structure
sec	Security scanner findings, vulnerabilities	SecApplicationRecord	shared structure
jh (downstream)	JiHu-only data	JhApplicationRecord	(downstream only)

The decomposition lives in config/database.yml.decomposed-postgresql (and decomposed-sec). Connection routing is automatic based on the model's base class.

Schema artifacts

• db/structure.sql (~2.8 MB) — authoritative DDL, regenerated by scripts/regenerate-schema.
• db/init_structure.sql (~2.4 MB) — bootstrap state for fresh installs.
• db/migrate/ — 591 forward migrations.
• db/post_migrate/ — 946 post-deploy migrations.
• db/schema_migrations/ — per-version digest files (~13K of them).
• db/gitlab_schemas/ — JSON schemas for cross-database constraints.

Migration types

Two migration paths are mandatory for online upgrades:

1. Pre-deploy migrations (db/migrate/) — schema changes safe to apply before code rolls out. Adds nullable columns, creates tables, adds indexes concurrently.
2. Post-deploy migrations (db/post_migrate/) — schema changes that require new code to be running. Drops columns, adds NOT NULL, removes indexes.

A migration must follow the migration style guide. The cop set under rubocop/cop/migration/ enforces:

• add_column_with_default instead of add_column ... default: (online-safe defaults).
• add_concurrent_foreign_key, add_concurrent_index.
• disable_ddl_transaction! for migrations that can't run in a transaction.
• Strict timeout settings.

Load balancing

gems/gitlab-database-load_balancing/ provides per-request connection routing:

• Reads default to a replica.
• Writes go to the primary.
• "Sticky" reads — after a write, subsequent reads on the same request go to the primary until replication catches up.

Workers declare data_consistency:

• :always — primary only.
• :sticky — primary if replication lag is high.
• :delayed — replica with lag tolerance.

Implementation: lib/gitlab/database/load_balancing/ and the gem.

Data isolation

gems/gitlab-database-data_isolation/ enforces per-database query isolation. Queries that cross databases are rejected at runtime; cross-DB references go through "loose foreign keys".

Loose foreign keys

A real FK can't span databases. Gitlab::LooseForeignKeys (lib/gitlab/database/loose_foreign_keys.rb) and config/gitlab_loose_foreign_keys.yml track which referencing rows must be cleaned up after a parent is deleted. A nightly worker enforces them.

Partitioning

Gitlab::Database::Partitioning (lib/gitlab/database/partitioning/) creates and drops Postgres native partitions on a schedule. Used for:

• audit_events (monthly time partitions).
• web_hook_logs.
• Several CI history tables.
• Some security findings tables.

The dynamic-partition schema is gitlab_partitions_dynamic (see the comment in db/structure.sql).

Background migrations

For large data backfills that can't fit in a single migration:

• lib/gitlab/background_migration/ defines re-entrant data migrators.
• Gitlab::BackgroundMigration::BatchedMigration records progress in the batched_background_migrations table.
• A scheduled worker drains them.

A typical pattern:

# In a post-deploy migration
queue_batched_background_migration(
  'BackfillUserDetailsFromUsers',
  :users,
  :id,
  job_interval: 2.minutes,
  batch_size: 1000
)

ClickHouse

For OLAP analytics, GitLab also writes to ClickHouse via lib/click_house/ (and gems/gitlab-active-context/ for embeddings). See ClickHouse.

Connection pooling

• Puma uses Postgres connections via the shared ActiveRecord pool.
• Sidekiq uses one pool per process; the data_consistency of each job decides whether to use the primary.
• PgBouncer is recommended in production (config/initializers/database_config.rb warns on missing prepared-statement settings).

Schema validation

Several scripts run in CI to keep the schema honest:

• scripts/validate_migration_schema — schema dump matches migrations.
• scripts/validate_migration_timestamps — no future-dated migrations.
• scripts/validate_name_collisions_between_migrations.
• scripts/validate_schema_changes.
• scripts/validate_loose_foreign_keys_ordering.
• gems/gitlab-schema-validation — schema parsing/validation.

Migration helpers

lib/gitlab/database/migration_helpers.rb is the workhorse module included into every migration. It provides:

• add_concurrent_index, remove_concurrent_index.
• add_concurrent_foreign_key, remove_foreign_key_if_exists.
• add_text_limit, remove_text_limit.
• with_lock_retries for ALTER TABLE under contention.
• disable_statement_timeout, enable_statement_timeout.

Where to make changes

• New migration: bin/rails generate migration MyMigration ... then edit, then scripts/regenerate-schema.
• New ActiveRecord model: under app/models/<area>/. Always inherit from the right base class.
• Cross-DB query: stop. Refactor to a loose foreign key or a separate worker.
• New partitioned table: see lib/gitlab/database/partitioning/ and add a partition strategy file.

Related

• Architecture — where the DB sits.
• Sidekiq jobs — data_consistency semantics.
• ClickHouse — OLAP store.
• Geo — DB replication for secondaries.