AI Coding Agents
Security
Keeping test data private with coding agents
September 25, 2026 - 2 minute read
Test data privacy becomes a software delivery concern when coding agents can read fixtures, database snapshots, logs, and test output. A task may need realistic edge cases without needing real customer records. The safest workflow gives the agent the minimum data required to reproduce the behavior and keeps that data inside the approved environment.
The GDPR’s data minimization principle requires personal data to be adequate, relevant, and limited to what is necessary. The principle is useful beyond regulated EU workloads. It gives engineering teams a practical test for every field copied into a fixture or prompt.
Set a test data privacy boundary
Classify the data sources before an agent starts. Repository fixtures, generated records, anonymized samples, support attachments, and production snapshots have different risks. Document which sources are permitted, where they may be stored, and whether they may leave the network boundary.
Prefer purpose-built synthetic fixtures for unit and integration tests. Preserve the structural property that triggers the bug, such as string length, locale, relationship shape, or permission state. Remove names, contact details, credentials, account identifiers, and free-form text that the test does not exercise. NIST’s work on differentially private synthetic data also shows why useful synthetic data and privacy protection require evaluation rather than assumption.
Factory documents how code, prompts, telemetry, and integrations move in its privacy and data flows guide. Choose a Factory deployment and model path that match the data classification before granting an agent access.
Control coding agent access to test data
Limit the agent to the fixture directory, test database, and commands required for the task. Deny access to production credential files and raw exports. Use short-lived credentials for isolated test services, and prevent those values from being printed in shell traces or attached to CI artifacts.
Ask the agent to summarize record shapes instead of copying rows into its response. When a failure log can contain request bodies or database values, add redaction before the log reaches the agent. Factory’s agent safety and controls describes path restrictions, hooks, sandboxing, and Droid Shield as separate controls around agent execution.
Keep a human approval step for exceptions. A production-derived sample should have a documented owner, legal basis where applicable, retention period, and deletion path. Convenience is not enough justification.
Validate test data privacy after the change
Review the diff for unexpected records, snapshots, and golden files. Scan commits for secrets, then inspect generated test reports and uploaded artifacts. Secret detection will not identify every piece of personal or confidential business data.
Run the tests from a clean checkout using only declared fixtures. A passing test that depends on a developer’s local database is neither reproducible nor safely reviewable. Record the fixture generator and seed so another reviewer can recreate the case.
Finish by checking that the agent’s output, CI logs, and pull request description contain no sensitive values. The evidence should explain the shape of the test case without reproducing the protected data.
Further reading
Talk with Factory
Discuss your team’s software development, privacy, or deployment requirements.
