AI Coding Agents
Enterprise AI
Context compaction must preserve the next step
September 24, 2026 - 2 minute read
Context compaction reduces a long session to information an agent can carry forward. The engineering test is whether the agent can continue correctly. A short summary that loses the modified file, the failed approach, or the user’s constraint can create more work than it removes.
Useful retained state includes the current task constraint, files already changed, checks that passed or failed, and the next action. If a reviewer has rejected an approach, that decision must survive too. Without those details, the next turn may repeat work or undo a correction even when the summary reads fluently.
Context compaction needs more than a ratio
Factory's vendor-run December 2025 compression study tested hundreds of compression points across 36,611 messages from opted-in engineering sessions. A GPT-5.2 judge graded probe responses about recall, artifacts, continuation, and decisions. The reported overall scores were 3.70 for Factory, 3.44 for Anthropic, and 3.35 for OpenAI on a five-point scale, for the approaches tested at that time.
Factory's approach maintained a structured summary and merged newly truncated material into it. That design gives task state a stable place to live across repeated compactions. The published results associated it with stronger retention of technical details, without establishing a current ranking of every coding product.
The same results expose an important limit. Artifact tracking was the weakest dimension for all three methods. Factory scored 2.45, Anthropic 2.33, and OpenAI 2.19. A session that produces a fluent summary can still lose track of which files changed.
Treat the working tree, patch, and test output as independent evidence. Before resuming a consequential edit, reconcile the summary with the repository. Avoid asking the agent to reconstruct exact file state entirely from memory.
Compression ratio belongs beside continuation quality in an evaluation. Reducing the next request does not establish lower total cost if the agent must reread files, repeat failed attempts, or ask the engineer to explain an earlier decision again.
Context compaction should survive real working sessions
Chainguard’s case study provides a complementary customer account. One Droid session lasted two weeks, spanned six repositories, and built 80 packages. Josh Wolf described treating Droid like a colleague that remembers the ongoing discussion. The story shows why continuity matters in sustained work, but it does not measure token savings or isolate compaction as the sole cause.
Use a representative task that naturally crosses a context boundary. Before compaction, record the task constraint, files modified, validation status, rejected approach, and next action. Afterward, ask the agent to continue without restating those facts. Check the resulting work against the record.
Include a correction in the test. For example, a reviewer may reject an earlier approach after the agent has already summarized it. Verify that the new instruction replaces the stale decision instead of living beside it ambiguously.
Use a source that actually measures tokens and cost for the financial comparison. Factory's hosted Analytics API is separate from customer OpenTelemetry activity metrics, which do not include token or cost data. Provider or gateway billing may be the appropriate source for custom inference. Pair usage with successful completion, repeated reads, and human repair.
A good compaction result leaves the agent able to take the next correct step with less reconstruction. That is a more demanding target than generating the smallest possible summary.
