AI Coding Agents
Enterprise AI
Model routing has to account for the warm cache
September 24, 2026 - 2 minute read
Model routing compares more than the listed price of the next model. A long-running coding session has already accumulated repository context, tool results, and decisions. Moving that history to another model can require processing it again at fresh-input rates.
A warm cache contains previously processed context that a provider can reuse for a later request. Reuse depends on that provider's prefix-matching rules, cache lifetime, model, and billing rates. A lower-priced model may cost more on the next turn if switching means paying to process the accumulated history again.
Model routing depends on the state of the task
The harness constructs the request and executes the resulting tool calls. Factory’s August 24 routing research describes using session history, cache state, tool results, and task outcomes to inform model choices. Repeated failed fixes can affect the next choice. The publication does not specify a threshold that guarantees a switch after a particular number of failures.
The published research gives an unusually useful counterexample to automatic cost reduction. A Prisma migration session remained on the frontier model for all 67 turns and saved nothing against the frontier-pricing baseline. The work did not offer a safe point to step down. A router that always chooses a cheaper model would miss that trade-off.
A separate worker can change the economics. The parent may preserve its warm context while a bounded worker receives only a focused assignment. That creates another session with its own cost, so the evaluation must include both sessions and the work needed to combine their results.
Cache behavior also changes after an idle period or context compression. Treat switching as a decision tied to the current request and remaining work, rather than a permanent rule that one model is cheaper for one task label.
Model routing results need a named baseline
The August research reported 58% lower aggregate production cost against pricing the same observed work as frontier-model calls. Its latency comparison used routed and frontier-pinned session cohorts, with median turn latency of 49 and 81 seconds respectively. That cohort comparison is different from repricing the same observed calls for the cost baseline.
The current Factory Router documentation reports 63% aggregate production savings against frontier-model pricing. The earlier 43% documentation figure is no longer current. The August research and current documentation describe separate published measurements, not a single result that can be substituted across dates or workloads.
The research also reports benchmark pass rates relative to a frontier-only baseline. “99% of the frontier model’s pass rate” on Terminal-Bench 2 means a relative result. It does not mean the router solved 99% of all benchmark tasks. Production quality comparisons have additional limitations because customers and tasks differ.
For an enterprise pilot, retain the actual billed cost, cache reads and writes, selected models, failed attempts, and accepted changes. Compare cost per successful task and elapsed time under the same acceptance criteria. Include reruns instead of removing them as inconvenient outliers.
Factory Router operates under Enterprise Controls. Confirm that the allowed model set and deployment requirements fit the pilot. The cited production study does not establish equivalent savings for a custom on-premises model fleet.
The practical buying question is whether routing reduces the cost of work the team accepts. A lower advertised token price helps only when the complete session, its cache behavior, and its validation support the decision.
