Section 9

Measuring success

Treat the control plane as a product with an owner, a plan and a scorecard. Report the scorecard monthly to the executive committee and quarterly to the AI governance board.²⁹

Metric	Definition	Target	Data source
Agent coverage	Share of known agents registered with an owner and a risk tier	Up	Agent Registry and Scanners
Governed traffic	Share of LLM and MCP calls routed through the AI gateway	Up	AI Gateway
Cost per outcome	Model spend per resolved task, by use case	Down	LLM Governance Business systems
AI cost visibility	Share of model and tool spend attributed to an owner and a use case within a day, and share of agents under a token-based rate limit	Up	AI Gateway and LLM Governance Billing data in FOCUS format
Value realized	Run-rate value of live use-case slices against their business cases, agreed with finance	Up	Business-case tracking Use-case KPIs
Policy enforcement	Violations blocked before impact versus found afterward, including exposure-limit breaches refused by a custom aggregate-risk policy where one is in place	Up	AI Gateway policy events Gateway aggregate-risk refusals
Time to contain	Minutes from alert to kill switch or credential revocation, and to open work handed to a named person	Down	Agent Kill Switch Identity logs Takeover drill records
Evaluation pass rate	Agents meeting quality thresholds before promotion	Up	Evaluation results recorded in the registry
Oversight coverage	Share of tier 3–4 high-risk actions with approval or audit	Up	Trusted Agent Identity Audit logs
Oversight effectiveness	Share of high-risk actions executed exactly as approved and within the approval’s validity window, with drills showing people can stop an agent in time	Up	Gateway approval-binding events Kill Switch drill records
Crypto readiness	Share of the asymmetric-crypto inventory with a migration plan	Up	Cryptographic inventory

The scorecard’s metrics are outcomes a board can read, and they move slowly. Delivery teams also need early signals that move within weeks, defined so that every team computes them the same way. Those signals should follow the agent’s whole trajectory as well as its final answer, checking that it chose the right tools, in the right order and at an acceptable cost.⁶⁰ They should also measure the value delivered, because adoption alone says little. In BCG’s 2026 survey of close to 12,000 employees and leaders, the share whose organizations had put agents into workflows rose from 13% to 30% in a year, yet half said their companies lack clear governance for teams of people and AI.⁸⁸

Scorecard metric	Leading indicator	How to measure it
Agent coverage	Unregistered agents found	Agents found by scanners but not yet registered, and days from discovery to registration
Governed traffic	New agents launched behind the gateway	Share of newly promoted agents whose first production call went through the gateway
Cost per outcome	Cost per successful task	Model and tool spend divided by tasks that passed their success check, by use case⁶⁰
AI cost visibility	Spend attributed within a day	Share of the previous day’s model and tool spend tagged to an owner and a use case
Value realized	Acceptance rate	Share of agent outputs accepted without significant edits, with undo and revert counted as rejections⁶⁰
Policy enforcement	Near misses	Calls blocked per 1,000, by policy, and their trend, with exposure-limit refusals counted separately
Time to contain	Drill time	Minutes to stop an agent and hand its open work to a named person in the latest drill
Evaluation pass rate	Trajectory quality	Tool-selection accuracy and plan adherence on sampled production traces⁶⁰
Oversight coverage	Verification latency	Time from an agent completing a task to a person approving it.⁶⁰ Very short times can mean approval without review.
Oversight effectiveness	Takeover rate	How often a person has to step in and take over a task the agent started⁶⁰
Crypto readiness	Algorithm agility	Share of gateways and service meshes that can change algorithms by configuration