How to catch performance regressions before merge

To catch a performance regression before merge, run the same representative workload against the branch and its merge base under identical conditions, compare the result with an agreed budget for latency, queries, memory, or bundle size, and repeat the runs to measure variance. A finding counts only when the difference is larger than the baseline noise.

Performance regression review checks the application after a coding agent changes it, rather than measuring how quickly the agent writes code. Suppose a changed endpoint looks fast with a warm local cache but performs a database query for every returned item on a cold request. Reviewers need a comparable workload to establish whether the change violates the service's latency or query budget.

A performance budget turns that limit into a merge criterion. Google’s performance budget guidance describes budgets for quantities such as resource size, timing milestones, and rule-based scores. Backend services can apply the same principle to latency, throughput, memory, database calls, or queue depth.

Start from a comparable baseline

Choose a representative workload before reviewing the patch. Fix the dataset, concurrency, cache state, runtime version, hardware class, and number of samples. Compare the branch with its merge base under the same conditions. A result from a warm local cache cannot support a claim about a cold production path.

For the endpoint example, run the same request against the branch and merge base with equivalent cold-cache preparation. Preserve the number and shape of returned items, capture query counts and plans, and measure response latency across repeated runs. Repeat the warm-cache case separately. That distinguishes added database work from a change in cache hit rate.

Other changes need different measurements, such as bundle size for a frontend dependency or memory growth for a worker. Keep existing service-level objectives and budgets as the source of truth.

The metric should match the kind of change. The table lists common pairings and the artifact a reviewer can inspect for each.

Record natural variance. If repeated baseline runs vary more than the observed difference, the test cannot support a regression finding. Improve the harness or report the result as inconclusive instead of assigning false precision.

Preserve raw measurements and profiler output with the review. Aggregated scores alone can hide one expensive route, query, or interaction.

Review the changed execution path

Trace the request through the changed code to the repeated queries. A query inside an item loop is a credible risk, but profiling should establish whether it runs for the chosen workload. If batching removes the extra queries, repeat both cold and warm runs and verify that missing-item and authorization behavior remain correct.

Keep deterministic tools in the loop. Bundle analyzers, profilers, query planners, load tests, and browser measurement tools produce artifacts that another reviewer can inspect. The agent can connect those results to the diff and propose a focused fix.

Make the finding reproducible

A useful performance finding names the affected input, changed metric, comparison commit, environment, and command. It also points to the code path that explains the result. Screenshots without configuration or a single timing from a developer laptop do not provide enough evidence for a merge decision.

Run correctness tests after any optimization. Caching, batching, and deferred work can improve a benchmark while changing freshness or failure behavior. Preserve the original acceptance criteria and add a regression check where the repository can run it consistently.

How Factory makes performance findings reproducible

Factory is a platform for running a software factory, where agents called Droids take work from a signal to a reviewed pull request. For performance, the useful Factory features are the ones that run the same comparison every time and tie the result to the diff.

Run the benchmark headless. Droid Exec runs a task as one non-interactive pass and can write JSON output with -o json, which keeps the comparison machine-readable for CI.

Review with performance instructions. The local /review command accepts custom review instructions, so a team can name the hot path, budget, and benchmark command. Findings come back with P0 to P3 severity and file locations. In CI, automated code review applies a review-guidelines skill to every pull request. Keep noisy benchmarks advisory until their variance is understood, and promote stable measurements to required checks.

Keep the environment steady. Droid Computers keep installed packages, files, and running services between sessions, which helps a repeated benchmark run against the same setup.

To set up both review modes, run these commands in the Droid CLI.

Frequently asked questions

What is a performance regression review?

A performance regression review checks whether a change makes the application slower or more expensive than an agreed budget, using a workload that compares the branch with its merge base under the same conditions.

What is a performance budget?

A performance budget is a limit that becomes a merge criterion. Web budgets cover resource size and timing milestones, and backend services apply the same idea to latency, throughput, memory, database calls, or queue depth.

How do you avoid false performance findings?

Measure the natural variance of repeated baseline runs. If the variance is larger than the observed difference, report the result as inconclusive and improve the harness.

How does Factory help with performance review?

Droid Exec can run the comparison headless in CI with JSON output, the local review command accepts custom performance instructions, and automated code review applies the same guidelines on every pull request.

Further reading

• Database query plan review
• Memory leak testing
• Test impact analysis
• Performance budgets 101