Factory Private
Data Governance
AI agent telemetry inside private networks
September 18, 2026 - 2 minute read
AI agent telemetry should answer a specific operational question. A team investigating slow validation may need execution timing and CI results. It may have no reason to retain the source code or complete conversation behind the task.
Decide what the organization needs to measure before enabling detailed collection. Activity metrics, raw content, and approval records have different uses and different sensitivity.
AI agent telemetry separates metrics from content
Factory's telemetry documentation describes metrics-only customer export by default. Optional message-content spans go to a customer-configured collector. The hosted Analytics API is a separate source.
Its privacy reference shows where each data type can go.
Screenshot of the linked telemetry privacy documentation, captured September 18, 2026.
Raw content export is off by default and has no automatic redaction. If enabled, it may include prompts, file contents, command arguments, and tool results. Approve the collector for those data classes before collecting them.
Set AI agent telemetry policy centrally
Organization-managed settings can pin the customer collector and content policy. Project, folder, and user copies of the telemetry block are ignored. Verify the effective configuration on the actual execution account.
Keep collector credentials out of broadly readable settings. Factory supports environment-variable references so the actual credential can arrive through the managed environment.
Test a missing credential and an unreachable collector. A dashboard receiving data from some machines may conceal another runner that never exported successfully.
Review downstream destinations too. A customer-operated collector can forward information to another service. Its storage, access, and retention policy belong in the same boundary review.
Choose identity detail deliberately
Aggregate granularity removes specified user and change identifiers and prevents message-content export. Some session, model, and workload dimensions remain. Review those fields before calling the dataset anonymous.
The export format affects trace availability. Factory documents no spans for the legacy format in aggregate mode, while the GenAI format retains trace structure with content attributes removed.
Inspect a sample from a non-sensitive task. Confirm that it matches the approved schema and use that sample as a check when changing collector transformations.
In an airgap, only the customer collector exists. Without a usable endpoint, the pipeline does not run. Plan an internal way to detect missing evidence rather than assuming a hosted fallback.
Connect monitoring to accepted work
Tool counts establish activity, not whether a change was correct. Use CI results, source-control history, and reviewer approval to establish the delivered outcome.
The customer OTEL metric set does not include token and cost data. Obtain inference cost from an appropriate provider, gateway, or infrastructure record. Label allocation assumptions when reporting estimates.
Droid exports metrics with delta temporality. Sum the new activity over the reporting window rather than treating the latest export as a cumulative total.
For a pilot, track accepted changes, validation failures, reviewer corrections, and operator intervention. Keep definitions stable so a configuration change does not look like a productivity improvement simply because measurement changed.
The result should be enough evidence to operate and review the workflow, with no unnecessary expansion of the sensitive data retained.
