Comparisons
AI Coding Agents
Factory and Claude Code: compare completed work
September 24, 2026 - 2 minute read
Factory and Claude Code can produce different bills for the same engineering request even when both deliver a working patch. The difference can come from model selection, cache reuse, tool output, context management, or failed attempts. A comparison needs to identify which of those changed.
As of September 21, 2026, Claude Code’s cost documentation describes controls for context management, model selection, and spending. Factory documents usage through its telemetry reference. Those sources establish ways to inspect cost. They do not establish a winner on an untested workload.
Compare Factory and Claude Code on the same result
Start from the same repository revision, environment, and task description. Set the acceptance criteria before either run. A patch that passes a narrow unit test but breaks the requested behavior should not count as a cheaper success.
Use the comparison record to separate the product experience from a same-model harness test. The former can use each product’s recommended configuration. The latter should hold the model and settings constant where both products support the configuration.
Record the exact model, provider, and reasoning settings so a model change does not get mistaken for a harness improvement. Separate input, output, cache reads, and cache writes because equal token totals can produce different bills.
Keep the acceptance tests and reviewer decision with the usage record. An incomplete patch does not count as a successful low-cost result. Include reruns, corrections, and human intervention so the first attempt does not hide the eventual cost.
Repeat the task set across more than one kind of work. Include a bounded fix, a repository investigation, and a change requiring sustained context. Keep failures in the denominator and report the task-level results, rather than only the best demonstration.
Factory and Claude Code expose different decisions
Factory’s custom-model documentation describes connecting provider keys, open models, and local endpoints in the CLI and desktop app. That flexibility makes model choice an explicit variable. It does not mean every endpoint has identical tool behavior or context limits.
Factory Router adds another possible configuration by selecting models during work. Compare a routed run separately from a pinned-model run. Otherwise, a claimed harness difference may actually be a routing decision.
Claude Code also has documented model configuration. Do not describe it as lacking cost controls simply because the controls differ. Capture the settings used and date the result so a later product update can be evaluated fairly.
A useful headline metric is total measured model cost divided by accepted tasks. Report elapsed time and reviewer effort alongside it. Treat subscription charges and metered API usage separately, because a usage display is not necessarily the amount an organization pays.
A demonstration video can suggest a test worth repeating. Without its repository state, task criteria, model settings, and usage accounting, it cannot support a reproducible numerical comparison. Publish a savings claim only when the underlying record makes the comparison inspectable.
