Self-improving software architecture in practice

September 24, 2026 - 5 minute read

A coding agent can recover from a failed command and still repeat the same mistake tomorrow. Self-improving software architecture has to account for what survives the session. The correction might belong in application code, a test, a tool definition, or a reusable instruction. Each has a different owner and a different path to production.

Factory's published software-factory model connects incoming engineering signals to implementation, validation, shipping, and monitoring. It describes model independence, sovereign intelligence, and continual learning as requirements for the broader system. Factory is the product used to operate that software delivery model. The practical design question is where a lesson becomes a controlled change, rather than another observation that the next session must rediscover.

Specify what self-improving software can change

Start by naming the artifact that should improve. A missing timeout in an application requires a code change. An agent repeatedly choosing the wrong test command may need corrected repository instructions. A tool that hides the reason for a failure may need a better response format. Replacing the model is a separate intervention, with different costs and evaluation requirements.

Arize's explanation of self-improving agents examines changes to behavior and persistent context, including improvements that do not involve model weights. That distinction is useful when assessing a proposed architecture. Ask which versioned artifact changes, which future runs will consume it, and how the team can restore the previous version.

For example, suppose a maintenance workflow edits generated files instead of their source. A repository instruction can identify the generator input and the regeneration command. A test can detect an inconsistent generated output. Both changes can be inspected in a pull request. A promise that the agent will remember to be more careful gives the reviewer much less to verify.

Factory's AGENTS.md documentation provides a concrete place for durable project commands and boundaries. Put a verified correction there when it should apply across repository work. Keep the change narrow enough that it does not teach unrelated workflows an exception they should never follow. The goal is a reusable correction with a known scope.

Give self-improving software a durable record

The record that explains a failure and the instruction that prevents its recurrence serve different purposes. The first preserves evidence. The second tells a later run what to do. Keeping both makes a correction easier to review and easier to retire when the repository changes.

Factory skills package reusable procedures in SKILL.md files with optional supporting material. They fit a lesson that applies to a specific workflow, such as verifying a database migration or checking a release. Always-on project instructions should remain short. A detailed procedure can live in a skill without becoming background text for every unrelated task.

Consider a deployment check that must wait for a preview environment to become ready. The failure record should identify the environment state and the evidence that the check ran too early. The reusable procedure should describe the readiness condition and what to report when it never arrives. Copying a transient error into instructions without that explanation risks preserving an accident as policy.

Record the reason for the change in the pull request and keep the resulting file in version control. A reviewer should be able to determine whether the guidance still applies after a tool, environment, or repository layout changes. Avoid treating every user correction as a permanent rule. Some corrections reflect a one-time request, and some apparent failures come from missing permissions rather than defective instructions.

Separate the evidence from permission to act

A system that can observe a problem does not automatically have authority to fix every resource involved. An alert may contain a production hostname, a customer report, or an instruction pasted from an untrusted source. Those details can support an investigation without becoming commands that the agent is authorized to execute.

Factory's agent safety and controls documentation distinguishes deterministic controls from model behavior. It documents command controls, managed hooks, and sandbox boundaries. Repository prose can explain the intended workflow, while configured controls determine which operations may run. A suggested instruction update should not be able to expand its own enforcement boundary.

Apply the same separation to the improvement process. A maintenance agent may propose an additional test or a correction to a skill. Changes to credentials, network access, branch protection, or required checks should have an explicit policy owner. Otherwise, removing the obstacle can become easier than satisfying the requirement the obstacle was meant to enforce.

The data boundary matters too. Preserve enough context to reproduce a failure without copying secrets or unnecessary customer content into a shared instruction file. Decide where the original evidence can be accessed, who can review it, and what sanitized explanation belongs with the code change. These choices should follow the organization's existing access and retention policies, rather than an assumption that all useful context belongs in agent memory.

Test the changed behavior beyond the original incident

A correction should address the observed failure without making neighboring work worse. If an instruction fixes one package by hard-coding its directory, test that it does not send every other package to the same location. If a tool response becomes shorter, check that it still includes the information needed to distinguish an empty result from a failed request.

Anthropic's work on improving tools for agents uses evaluation to guide changes and describes held-out test sets to check whether gains extend beyond the development examples. The applicable engineering habit is to evaluate cases beyond the examples used to develop the correction. A replay of the original incident is necessary evidence, but it is a narrow slice of future work.

Factory's Automated QA documentation describes checks through web, CLI, and API surfaces, with recorded evidence. It also documents failure-learning options. The default suggests changes in the report for manual review. Other options can commit updates or open a draft pull request. Choose the mode deliberately, because a proposed lesson and an accepted lesson have different consequences.

Keep the acceptance condition outside the agent's account of its own success. The relevant command should return the expected result, the application should behave correctly, and the required checks should remain enabled. When a test cannot run because an environment is unavailable, preserve that result as blocked. Calling it a pass would turn missing evidence into a false improvement.

Keep internal research and product guarantees distinct

Factory's January 2026 Signals research gives a concrete example of an internal improvement process. It describes analyzing session patterns, identifying recurring friction, creating engineering work, and having Droid prepare fixes. The report retains human approval before a pull request merges.

That account supports a bounded claim about an internal system. It does not establish that every customer workflow receives the same analysis, that models retrain on customer sessions, or that an arbitrary repository will improve automatically after deployment. An architecture decision should identify the documented customer mechanisms it uses and the parts the team still needs to configure.

For a first implementation, choose a recurring problem whose correction can be expressed in a small, reviewable artifact. Identify the evidence source, the file or tool that may change, the acceptance test, and the person who can approve the result. Retain the old version so the team can undo an unsuccessful lesson.

The architecture becomes useful when a later run can demonstrate the changed behavior without reconstructing the original conversation. A reviewer should be able to inspect what persisted, explain why it belongs at that scope, and remove it when the evidence no longer supports it. That is a more actionable requirement than asking an agent to improve itself without specifying what improvement means.