AI Coding Agents
Enterprise AI
Model routing has to account for the warm cache
September 24, 2026 - 2 minute read
Model routing compares more than the listed price of the next model. A long-running coding session has already accumulated repository context, tool results, and decisions. Moving that history to another model can require processing it again at fresh-input rates.
Factory’s August 24 routing research reported 58% lower aggregate production cost against pricing the same observed work as frontier-model calls. Median turn latency was 49 seconds for routed sessions, compared with 81 seconds for frontier-pinned sessions. These are Factory-reported results with different cost and latency comparisons, not an independent promise of savings.
Model routing depends on the state of the task
The harness constructs the request and executes the resulting tool calls. It can observe whether a patch passed its tests, whether the same error keeps recurring, and whether the next job needs the full session history. Those signals help distinguish an inexpensive mechanical step from a diagnosis that merits stronger reasoning.
The published research gives an unusually useful counterexample to automatic cost reduction. A Prisma migration session remained on the frontier model for all 67 turns and saved nothing against the frontier-pricing baseline. The work did not offer a safe point to step down. A router that always chooses a cheaper model would miss that trade-off.
A separate worker can change the economics. The parent may preserve its warm context while a bounded worker receives only a focused assignment. That creates another session with its own cost, so the evaluation must include both sessions and the work needed to combine their results.
Cache behavior also changes after an idle period or context compression. Treat switching as a decision tied to the current request and remaining work, rather than a permanent rule that one model is cheaper for one task label.
Model routing results need a named baseline
The current Factory Router documentation separately reports 43% aggregate production savings against top-tier pricing. That figure and the August research result are distinct published measurements. Neither should be presented as a guaranteed saving or silently substituted for the other.
The research also reports benchmark pass rates relative to a frontier-only baseline. “99% of the frontier model’s pass rate” on Terminal-Bench 2 means a relative result. It does not mean the router solved 99% of all benchmark tasks. Production quality comparisons have additional limitations because customers and tasks differ.
For an enterprise pilot, retain the actual billed cost, cache reads and writes, selected models, failed attempts, and accepted changes. Compare cost per successful task and elapsed time under the same acceptance criteria. Include reruns instead of removing them as inconvenient outliers.
Factory Router operates under Enterprise Controls. Confirm that the allowed model set and deployment requirements fit the pilot. The cited production study does not establish equivalent savings for a custom on-premises model fleet.
The practical buying question is whether routing reduces the cost of work the team accepts. A lower advertised token price helps only when the complete session, its cache behavior, and its validation support the decision.
