# Factory and Claude Code: compare completed work

> Compare Factory and Claude Code token efficiency using accepted changes, billed usage, cache behavior, and reruns. Keep model and harness effects separate.

URL: https://factory.com/articles/factory-claude-code-token-efficiency
Published: 2026-09-24
Updated: 2026-10-29

Factory and Claude Code can produce different bills for the same engineering request even when both deliver a working patch. The difference can come from model selection, cache reuse, tool output, context management, or failed attempts. A comparison needs to identify which of those changed.

As of September 21, 2026, Claude Code’s cost documentation describes controls for context management, model selection, and spending. Factory documents usage through its [telemetry reference](https://docs.factory.com/enterprise/telemetry/data-reference). Those sources establish ways to inspect cost. They do not establish a winner on an untested workload.

## Watch Sarah Chieng's comparison

Sarah Chieng's [video comparison](https://www.youtube.com/shorts/Q1kB6li2KnU) gives this distinction a concrete example. Her [original post](https://x.com/MilksandMatcha/status/2091920183812329904) describes using Opus 4.6, the same initial prompt, and an empty repository in Claude Code and Factory Droid to build an Excalidraw-style application.

<iframe
  width='100%'
  height='400'
  src='https://www.youtube.com/embed/Q1kB6li2KnU'
  title='Sarah Chieng compares Claude Code and Factory Droid'
  frameBorder='0'
  allow='accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture'
  allowFullScreen
  className='mb-8 aspect-video w-full rounded-lg'
/>

This is a creator demonstration shared by Factory, not an independently reproduced benchmark. Holding the model and initial request constant makes the surrounding execution workflow worth examining. It does not establish that every other setting or execution condition was identical.

Use the demonstration to decide what to inspect in your own trial. Compare the working application, the checks performed, any additional guidance, and the usage record together. A result from one build should not become a general token-savings percentage without repeated tasks and comparable accounting.

The procedure below provides a way to make that follow-up comparison.

## Compare Factory and Claude Code on the same result

Start from the same repository revision, environment, and task description. Set the acceptance criteria before either run.

A patch that passes a narrow unit test but breaks the requested behavior should not count as a cheaper success.

Use the comparison record to separate the product experience from a same-model harness test. The former can use each product’s recommended configuration. The latter should hold the model and settings constant where both products support the configuration.

Factory has published one same-model test of this kind. In its October 2026 [Droid 1.0 research post](https://factory.com/news/frontier-harness), Factory ran Droid and `claude-code@2.1.306` on all Terminal-Bench 4 tasks with Claude Opus 5.5 at high reasoning, three trials per task, between October 23 and 28, 2026. Factory reports Droid scoring 7.6 points higher (65.2% against 57.6%), taking 26% less time per passed task, and using 17% fewer tokens. Cost and time count only the tasks that both harnesses passed in all three trials, at list price.

Those are vendor-published results on terminal tasks. A team's own trial record still decides its workload.

Record the exact model, provider, and reasoning settings so a model change does not get mistaken for a harness improvement. Separate input, output, cache reads, and cache writes because equal token totals can produce different bills. Keep the acceptance tests and reviewer decision with the usage record. An incomplete patch does not count as a successful low-cost result. Include reruns, corrections, and human intervention so the first attempt does not hide the eventual cost.

Repeat the task set across more than one kind of work. Include a bounded fix, a repository investigation, and a change requiring sustained context. Keep failures in the denominator and report the task-level results, rather than only the best demonstration.

## Factory and Claude Code expose different decisions

[Factory’s custom-model documentation](https://docs.factory.com/model-independence/byok) describes connecting provider keys, open models, and local endpoints in the CLI and desktop app. That flexibility makes model choice an explicit variable.

It does not mean every endpoint has identical tool behavior or context limits.

[Factory Router](https://docs.factory.com/model-independence/factory-router) adds another possible configuration by selecting models during work. Compare a routed run separately from a pinned-model run. Otherwise, a claimed harness difference may actually be a routing decision.

Claude Code also has documented model configuration. Do not describe it as lacking cost controls simply because the controls differ. Capture the settings used and date the result so a later product update can be evaluated fairly.

A useful headline metric is total measured model cost divided by accepted tasks. Report elapsed time and reviewer effort alongside it. Treat subscription charges and metered API usage separately, because a usage display is not necessarily the amount an organization pays.

A demonstration video can suggest a test worth repeating. Without its repository state, task criteria, model settings, and usage accounting, it cannot support a reproducible numerical comparison.

Publish a savings claim only when the underlying record lets readers inspect the comparison.

## Frequently asked questions

### Why can two coding agents bill differently for the same patch?

The difference can come from model selection, cache reuse, tool output, context management, or failed attempts. A comparison needs to identify which of those changed.

### Is the Sarah Chieng video a benchmark?

No. It is a creator demonstration shared by Factory, using the same model, initial prompt, and empty repository in both tools. It is not an independently reproduced benchmark.

### What metric should a token efficiency comparison report?

Total measured model cost divided by accepted tasks, with elapsed time and reviewer effort. Separate input, output, cache reads, and cache writes, and include reruns and corrections.

### Has Factory compared Droid and Claude Code on the same model?

Yes. On Terminal-Bench 4 with Claude Opus 5.5 at high reasoning, run October 23 to 28, 2026, Factory reports Droid scoring 7.6 points higher than Claude Code, taking 26% less time per passed task, and using 17% fewer tokens. These are vendor-published results.

### Should routed and pinned-model runs be compared together?

No. Factory Router selects models during work, so compare a routed run separately from a pinned-model run. Otherwise a claimed harness difference may be a routing decision.

## Further reading

- [Factory routing research](https://factory.com/news/model-routing-belongs-in-the-harness)
- [Factory context-compression research](https://factory.com/news/evaluating-compression)
- [Factory deployment patterns](https://docs.factory.com/enterprise/network-and-deployment)