Agent Observability for Copilot: The New Capture Controls Are Not One Policy Across Your Clients

On 12 October GitHub gave enterprise administrators six separate switches for what Copilot clients put into exported OpenTelemetry data: identity, prompts, responses, toolArguments, toolOutput and policyDetail, all under telemetry.capture in managed settings and all off by default. The pitch is agent observability without a content dragnet: attribute each agent session to a GitHub login, a local OS account and a machine name, and collect nothing else.

GitHub documents the full set as working in Copilot CLI, the GitHub Copilot app and VS Code sessions running on Agent Host. Outside those, and in places inside them, the switches are not the policy you think you set. The documentation for this release lives in two places, GitHub's and Microsoft's, and on two points they disagree.

What the docs commit shows about agent observability defaults

The docs change landed as a single commit to github/docs on 13 October. Reading its diff rather than the published page turns up three things the announcement leaves out.

First, the meaning of an existing key widened. Before the commit, the reference described captureContent as including prompt and response content. After it, captureContent is legacy shorthand for all five content classes together, which per the same page includes raw tool arguments and commands, tool results and error details, and unbounded sandbox path lists. Whether client behavior changed or the old text was incomplete, the diff cannot say. Either way, an enterprise that switched captureContent on to read prompts was, by today's reference, also collecting the commands its agents ran and what came back from them. VS Code's settings table still labels the setting as prompt and response content, while the content-capture section of the same page says it fills in tool schemas, arguments and results.

Second, a guarantee got softer. The old concepts page said managed OTel settings are enforced and cannot be overridden. The new text says explicit managed values take precedence. A new lockCaptureContent flag exists to stop local settings switching on a content class that the managed object leaves unset. In other words, without the lock they can.

Third, the GitHub Copilot app moved from unsupported to supported for telemetry in the client table.

The assumption this breaks: one managed block, one policy

A platform team reading the changelog may plan on writing one telemetry object and getting one collection posture across the fleet. Four documented facts break that.

VS Code's local harness is still all or nothing. GitHub's reference and its monitoring how-to both say the local agent harness honors identity and the legacy captureContent, but not the five individual classes. If you need tool arguments from a developer on the local harness, you also get their prompts.

The two vendors disagree on environment variables. GitHub's reference says managed telemetry values beat local settings, SDK configuration and environment variables. VS Code's guide says that in the Copilot Chat extension, OTel environment variables can still override managed values. Its exceptions are managed identity capture and managed resource attributes. For everything else it tells you to remove conflicting variables from managed devices yourself.

They also disagree on Agent Host identity. GitHub's monitoring how-to says identity capture works in both local and Agent Host sessions in VS Code. VS Code's guide says identity currently applies only to the local harness, and that Agent Host telemetry does not carry those attributes.

Managed telemetry objects replace each other, they don't merge. Settings are deployed by MDM, by the server from a .github-private repository, or as a file, in that order of precedence. Sandbox and permission rules compose in the most restrictive direction across sources. telemetry is not on that list: the highest-precedence source supplies the entire object. If MDM defines telemetry and leaves out capture, the capture block your security team committed to .github-private does not apply.

JetBrains is listed as supporting telemetry, but it is absent from the list of clients with fine-grained capture. Neither page says what JetBrains does with a capture object. And the Copilot CLI changelog, through 1.0.97 on 13 October, has no entry for the capture classes, so there is no version to pin a rollout to.

What to do differently this quarter

The decision to change: do not turn on identity fleet-wide as the first, low-risk step. Run it as a measured pilot on machines whose harness you control. Identity attaches a GitHub login to the existing pseudonymous enduser.pseudo.id, which GitHub says it leaves untouched. The OpenTelemetry semantic conventions classify that pseudonymous attribute as linkable personal data. Once both arrive in the same export, your collector holds the join key. If the pseudonymous ID is stable across sessions, which neither vendor's docs state, that also exposes data you collected before.

Before the pilot:

1. Put the whole telemetry object in one delivery channel. If MDM carries any part of it, MDM carries all of it.
2. Set lockCaptureContent: true and set every capture class explicitly, including the ones you want off.
3. Issue the collector an ingest-only credential. The headers value is a literal in the managed object, so every client that sends telemetry holds it.
4. Test the override on one VS Code machine. Set COPILOT_OTEL_FILE_EXPORTER_PATH=/tmp/copilot-otel.jsonl, which VS Code documents as writing all signals to a JSON-lines file, run one agent turn in each harness, and count what landed with grep -c gen_ai.input.messages /tmp/copilot-otel.jsonl. Any lines in the file mean a local variable redirected the export; a non-zero count with prompts off means the managed policy is not what governs that client.

This is the last-mile problem in miniature. The model isn't what goes wrong here. A policy file meets four clients, three delivery channels and two vendors' documentation. The schema side of tracing agents is covered in LLM observability is mostly a schema problem. For help deciding which classes justify their privacy cost, that judgment is what our LLM consulting work is for.

What would make this wrong

The central claim is that a single managed telemetry object does not produce a single collection policy across Copilot's VS Code harnesses. It is wrong if the step 4 test, run on the current VS Code release with capture.identity on and every content class off, shows identity attributes on Agent Host spans, no prompt content from the local harness, and an OTEL_EXPORTER_OTLP_ENDPOINT set by the developer that fails to move the data. If all three hold, the VS Code guide is stale, GitHub's reference is the truth, and the identity pilot can widen to the fleet without the harness caveats above.