AI Coding Agents
Reliability
How to test for memory leaks in agent-written services
September 28, 2026 - 4 minute read
To test for a memory leak, warm the service up, run the same bounded workload in several batches, and record heap and resident memory after each one. Steady growth across equivalent batches after warm-up is the signal, and heap snapshots from two batches show which objects are being retained.
Memory leak testing measures whether a service retains memory after repeated work should have released it. Agent-generated code can introduce long-lived listeners, timers, caches, closures, or request objects while still passing functional tests. A repeatable workload and an explicit memory budget make that risk visible before a change reaches production.
Heap snapshots are useful evidence, but they can pause a process and may require about twice the current heap while being created. The official Node.js heap snapshot guidance recommends care in production because snapshot creation can block the event loop or exhaust available memory. Capture them in an isolated test process whenever possible.
Design a stable memory leak test
Choose one bounded workload that exercises the changed path. Warm the service first so module loading, connection setup, and lazy caches do not look like leaks. Run the workload in several batches, release references, allow garbage collection under controlled test settings, and record memory after each batch.
Resident set size can reveal process growth, while heap measurements narrow the problem to managed memory. Track both when the runtime exposes them. A fixed threshold alone can be brittle across machines, so also compare the slope after warm-up. Continued growth across equivalent batches is stronger evidence than one high sample.
The test is a loop with a decision at the end. A clean run passes, and growth beyond the budget sends the investigation to heap snapshots.
Keep the environment stable. Use fixed fixture sizes, bounded concurrency, and local dependencies. Disable unrelated background jobs that add noise.
Isolate retained objects in agent-generated code
When the test detects growth, compare heap snapshots from after two equivalent workload batches. Look for object types whose retained count or size rises and never returns toward the baseline. Trace retaining paths back to changed code rather than treating the largest object as the cause.
Common causes include event listeners without cleanup, maps keyed by request data, promises retained by queues, unbounded telemetry attributes, and response bodies held after parsing. Add a regression test around the smallest behavior that reproduces the retention.
The common causes leave recognizable patterns in a snapshot comparison.
| Cause | What the snapshots show | Typical fix |
|---|---|---|
| Event listener without cleanup | Listener count rises with each request | Remove the listener in the scope that adds it |
| Map keyed by request data | Map size tracks request volume | Bound the map or add eviction |
| Promise retained by a queue | Pending promises never settle | Settle or cancel on timeout |
| Unbounded telemetry attributes | Attribute sets grow with input | Limit attributes to a fixed vocabulary |
| Response body held after parsing | Large buffers stay reachable | Drop the reference once parsed |
Native allocations and subprocesses may increase resident memory without appearing in the JavaScript heap. Match the diagnostic tool to the runtime and allocation source.
Give a coding agent measurable acceptance criteria
Provide the exact workload, batch count, warm-up behavior, metric, and acceptable growth. State whether the agent may expose garbage collection in the test process. Include a timeout so a stalled diagnostic run fails clearly.
Review memory leak test evidence
Store compact measurements in CI output and keep full snapshots as access-controlled artifacts when policy permits. Heap snapshots can contain application data, so treat them as sensitive. Do not attach them to public issues or pull requests.
Repeat a suspected failure before accepting it. A leak produces a consistent retention pattern. One noisy run calls for better isolation, not an arbitrary increase in the budget.
How Factory helps catch memory leaks
Factory is a platform for running a software factory, where agents called Droids take work from a signal to a reviewed pull request. For memory leaks, Factory adds a reviewer that looks for leak patterns in the diff, and gives the agent a stable place to run the measurement.
Code review on every pull request. Factory's automated code review focuses on clear bugs, and its published list includes resource leaks, async and await mistakes, and missing error handling. A review-guidelines skill can add repository rules, such as checking that every new listener, timer, or cache has a cleanup path.
Local review before pushing. The /review command in the Droid CLI reviews uncommitted changes, a commit, or a branch against its base, and accepts custom review instructions. Findings come back with P0 to P3 severity, file locations, and suggested fixes.
A stable environment for measurement. Droid Computers are persistent environments hosted by Factory that keep installed packages, files, and running services between sessions. A Droid can repeat the workload there instead of on a developer laptop, and the agent should return the test command and measurements as its result.
To try both review modes on your own repository, run these commands in the Droid CLI.
Frequently asked questions
How do you detect a memory leak in a service?
Run the same bounded workload in several batches after a warm-up and record heap and resident memory after each batch. Continued growth across equivalent batches is stronger evidence than one high sample.
Are heap snapshots safe to take in production?
The Node.js documentation advises care, because creating a snapshot can block the event loop and may need about twice the current heap. Capture snapshots in an isolated test process when possible.
What usually causes memory leaks in agent-written code?
Common causes are event listeners without cleanup, maps keyed by request data, promises retained by queues, unbounded telemetry attributes, and response bodies held after parsing.
What should a coding agent return for a memory leak task?
The exact test command, the batch measurements, and the acceptable growth it was checked against. A general statement that memory looks stable is not evidence.
How does Factory review changes for memory leaks?
Factory automated code review lists resource leaks among the bugs it looks for, and a review-guidelines skill can add rules about listeners, caches, and cleanup paths.
Further reading
Talk with Factory
Discuss your team’s software development, privacy, or deployment requirements.
Contact us
