Copilot's New Agent Observability Counts MCP Connections, Not Tool Calls

The Copilot usage metrics reports already covered CLI activity. On 17 September GitHub extended that coverage one level down, to the customizations themselves: five new arrays report which skills, custom agents, MCP servers, slash commands and plugins developers invoked, and five counters report how many distinct ones were in play. It gives an administrator agent observability at the level of the individual tool. It is also a narrower instrument than the field names suggest, in two ways the changelog entry lists under its closing notes and the reference documentation spells out.

What shipped

The new fields are totals_by_skill, totals_by_custom_agent, totals_by_mcp, totals_by_slash_cmd and totals_by_plugin, each an array holding at most five entries, ordered from highest to lowest interaction_count. Beside them sit distinct_skill_use_count, distinct_custom_agent_use_count, distinct_mcp_use_count, distinct_slash_cmd_use_count and distinct_plugin_use_count. They appear in per-user 1-day and 28-day reports, in aggregated 1-day reports, and inside each day_totals record of an aggregated 28-day report.

Access needs the Copilot usage metrics policy enabled. The REST reference puts the enterprise endpoints behind an enterprise owner, a billing manager, or the fine-grained View Enterprise Copilot Metrics permission, with the manage_billing:copilot or read:enterprise scope; the organization endpoints want an organization owner or View Organization Copilot Metrics, with read:org. Reports start from 10 October 2025 and stay available for up to a year from the current date. The new fields, though, can be null or absent while Copilot CLI customization data is still rolling out. A null is not a zero, and nothing on the page promises a backfill, so a year of history does not mean a year of these five arrays.

The MCP counter measures the handshake

Each category counts something different, and one of them is not what the field name implies. For skills, interaction_count counts invocations. For custom agents, it counts starts. For MCP servers, the reference states that it counts successful or failed connection and reconnection attempts, and that tool calls through an already connected server do not increase it.

The Model Context Protocol specification dated 2025-06-18 splits a session into three phases. Initialization is the client initialize request, the server response, and the client notifications/initialized. Operation is everything after that, including any use of the tools the server advertised through its tools capability. Shutdown closes the transport. So totals_by_mcp is a count of phase one.

The practical consequence runs the opposite way to intuition. An MCP server configured in every developer's client and never usefully called accrues an interaction on every CLI start and every reconnect. A server that carried two hundred tool calls inside one long-lived session accrues one. Ranking your MCP estate by interaction_count and cutting the bottom of the list can retire the servers that hold their connections rather than the servers nobody calls.

Agent observability with the names removed

The second limit is naming. Recognized first-party names can appear when available, but customer-defined identifiers are not exposed: skills, custom agents, MCP servers and plugins known only by a customer-specific hash are grouped under a single other entry in the array. Slash commands get their own label, custom.

GitHub's published example schema shows how much that removes. In the per-user example, totals_by_skill has exactly one entry, other, with five interactions. totals_by_plugin is also a single other row. totals_by_mcp lists github-mcp-server at eight interactions and other at three; totals_by_custom_agent lists general-purpose at four and other at two; totals_by_slash_cmd lists /plan at three and custom at one. Not one of the five arrays in that example is free of the unnamed bucket.

How much of a real deployment ends up in that bucket is measurable from outside GitHub. A code search over indexed public default branches returns 57,472 files named .mcp.json containing mcpServers, and 727 of those also mention api.githubcopilot.com. That is about 1.3 percent pointing at GitHub's own hosted server. Public repositories are not enterprises, .mcp.json is one of several places a client reads configuration, and private configuration may look different. But the ratio is not close, and an enterprise whose agent tooling was mostly written in-house should expect these arrays to resolve to one large other row and a short tail of GitHub's own names.

What changes for someone already pulling these reports

Three things, concretely. Do not add plugin and skill interaction counts together: every plugin interaction is already inside totals_by_skill, and only non-plugin skill activity is unique to it. Treat the distinct counts as the primary signal rather than the arrays, because they retain the full identifier cardinality including items outside the top five, and are therefore the only field that separates your own tooling into items rather than one bucket. And note that aggregated distinct counts are not sums of per-user distinct counts, so an organization figure cannot be reconstructed by adding up its members.

There is also no Enterprise Server answer here. The documentation is not versioned for it: /en/enterprise-server@3.22/copilot returns 404, while /en/enterprise-server@3.22/rest/actions at the same version returns 200.

The verdict

Adopt it for one job and ignore it for the other. The distinct counts are worth trending weekly, because a rising distinct_mcp_use_count against a flat headcount is evidence that tool sprawl is happening, and that is a governance question before it is a cost one. The arrays are not an inventory and should not be used to rank, retire or justify individual servers.

The falsifiable claim: for an organization whose agent tooling is predominantly internal, other will hold the majority of interactions in at least three of the five arrays. What would show it wrong is a report where named first-party entries carry most of the volume, which would mean the estate is mostly GitHub's own tooling and the naming limit costs nothing.

Check it on your own first report rather than taking either side on faith:

jq --arg o other '
  {other: ([.totals_by_mcp[] | select(.mcp == $o) | .interaction_count] | add // 0),
   total: ([.totals_by_mcp[].interaction_count] | add // 0),
   distinct: .distinct_mcp_use_count}' report.json

If other is above half of total, the array is a headline and the distinct count is the measurement. This is the same shape as the gap described in why enterprise AI doesn't ship and POC to production AI: the instrument that looked sufficient during the pilot turns out to measure adjacent to the thing you needed to know. Deciding which agent tooling becomes shared infrastructure, and what evidence justifies keeping it, is scoping work we do as part of AI agent development.