Copilot Metrics Got Cohorts. Now AI Adoption Has Fewer Places to Hide.

GitHub added AI adoption cohorts to the Copilot usage metrics API. That sounds like dashboard garnish until you look at what it measures: not just whether a developer touched Copilot, but which Copilot surfaces they used over a rolling 28-day window.

That matters because most AI rollout reporting is still stuck in the “licenses assigned” swamp. A license is not adoption. An active user is not maturity. And a chart that says “up and to the right” is not a strategy, even if someone put it in a quarterly business review and dimmed the lights.

GitHub’s May 29 changelog says the Copilot usage metrics API now exposes a user-level ai_adoption_phase field and an aggregate totals_by_ai_adoption_phase array for organization and enterprise reports. The classification is based on Copilot product usage across the last 28 days, with engagement requiring usage on at least two days.

What changed

GitHub now groups engaged users into four phases:

• Phase 0: No cohort — the user did not meet the engagement criteria for a phase.
• Phase 1: Code first — the user used code completion and/or IDE agent mode.
• Phase 2: Agent first — the user used one GitHub-based agent surface, such as Copilot cloud agent, Copilot code review, or Copilot CLI.
• Phase 3: Multi-agent — the user used two or more GitHub-based agent surfaces, or the GitHub Copilot app.

The API also includes a version field for the classification logic, starting with v1, which is the sort of boring detail that prevents future reporting archaeology. When the product surface changes, the definition can change without silently rewriting the past.

At the enterprise and organization levels, GitHub says the grouped metrics include engaged users, user-initiated interaction averages, code generation and acceptance activity averages, lines added/deleted averages, pull request activity averages, and median time-to-merge averages. GitHub’s Copilot usage metrics REST docs also make clear that access depends on enabling the Copilot usage metrics policy and using the newer usage metrics endpoints, not the old legacy metrics API.

Why builders and operators should care

This is not just a reporting feature. It is a quiet admission that AI tool adoption is now multi-surface and operationally messy.

A developer using autocomplete in the IDE is having a different workflow experience than one delegating review, CLI tasks, or cloud-agent work. Those are different risk profiles, training needs, governance questions, and productivity hypotheses. Treating them as one blob called “Copilot usage” is like measuring database health by asking whether Postgres is installed.

The useful shift here is cohort movement. If your organization has a large Phase 1 population and almost no Phase 2 or 3 usage, you probably have adoption of assisted typing, not agentic workflow change. That may be fine. It may even be the right place to stop for some teams. But it is not the same claim vendors make when they sell autonomous development workflows wearing a jetpack.

If the cohorts move from code-first to agent-first, you can ask better questions:

• Which teams are actually using agent surfaces, and for what work?
• Are agent-heavy teams seeing faster pull request flow, or just generating more review load?
• Does adoption correlate with healthier delivery metrics, or with noisier repositories?
• Where do enablement, documentation, and guardrails need to be different?

Those questions are more useful than “how many seats are active?” Active seats tell you whether the lights are on. Cohorts tell you which room people are standing in.

The practical takeaway

If you run engineering platforms, developer experience, security, or AI enablement, treat this as a prompt to separate rollout metrics into at least three buckets:

1. Access metrics — who has the tool.
2. Engagement metrics — who uses it with enough frequency to count.
3. Workflow metrics — which surfaces and behaviors are changing.

The third bucket is where this GitHub change helps. It gives platform teams a way to distinguish “developers are accepting completions” from “developers are adopting agent-based workflows.” That distinction matters for training plans, security review, budget justification, and expectation management.

It also matters for incident analysis. If a future bug, policy violation, or leaked context trail comes from an agent surface, you do not want the postmortem to say “Copilot was active.” You want to know which surface, which workflow, which cohort, and what controls existed around it.

Caveats

There are still sharp edges.

First, these are GitHub-defined phases. They are useful, but they are not a universal maturity model. They reflect Copilot’s product surfaces and GitHub’s current classification logic. The version field helps, but teams should still document what the cohort definitions meant at the time they used them.

Second, usage is not value. More Phase 3 users does not automatically mean better engineering outcomes. It may mean better workflows, or it may mean you have successfully taught everyone to summon a small distributed system inside their pull request process. Measure outcomes separately.

Third, the metrics depend on access, policy configuration, and available telemetry. GitHub notes the feature is available to enterprise administrators and organization owners with access to Copilot usage metrics through the REST API. If you build executive dashboards on top of this, make sure the data collection assumptions are visible. Hidden assumptions are where dashboards go to become folklore.

Bottom line

GitHub’s cohort metrics are a small API change with a bigger operational message: AI adoption should be measured by workflow, not vibes.

That does not make the numbers magical. It just makes them less useless, which is a meaningful step forward in a field where “adoption” too often means “we bought the seats and everyone clapped politely.”