Section 4

Defining requirements and security

Requirements turn the use case into what the solution must do and the conditions it must meet. Security requirements set how every actor is identified, what every call may do and how people stay able to intervene. Both are written with the client, prioritized, and traced to a KPI and a phase so the plan and the scorecard can be checked against them.

Requirement types

Writing good requirements

• One requirement per row, in plain language, with a measurable acceptance test.
• A priority: Must, Should, Could or Won’t (this phase).
• An owner on the client side who can accept it.
• The phase in which it must be met, which keeps early phases small.
• The KPI it moves or protects, so the scorecard traces back to it.
• Where it came from (workshop, policy, regulation), so it can be defended.

Security by layer

Use the nine layers of the handbook’s reference use cases (template T5). Security follows zero-trust principles: every actor is authenticated, every call is authorized for the specific task, and nothing is trusted because of where it sits on the network.⁹⁰

A vendor-neutral identity checklist

Identity underpins every other control, and standards work now gives it a vendor-neutral shape. NIST’s National Cybersecurity Center of Excellence has proposed a project on software and AI agent identity and authorization, focused on identifying, authorizing and auditing agents, on non-repudiation, and on controls against prompt injection.⁴⁵ An IETF draft on AI agent identity builds on workload identity, short-lived credentials bound to a key, OAuth delegation and token exchange, and real-time revocation, and it calls static API keys “an antipattern for agent identity.”⁴⁶ Both are drafts and will change, so present them as emerging. Together they give a checklist for T4 and T5:

• A unique identity for every agent instance, never a shared service account.
• Delegated authority that narrows at each hop, carried in on-behalf-of tokens.
• Auditable actions: who or what acted, on whose behalf, under which approval.
• No static API keys. Short-lived credentials only.
• Revocation that takes effect in real time, which gives the kill switch a second path.

The platforms are converging on the same pattern: Microsoft Entra Agent ID gives each agent an identity of its own, and Google distinguishes agents that act as their user, as themselves or under delegated authority.^12,47

Mapping threats to controls

Security teams judge a design against the threat lists they already use. The OWASP Top 10 for Agentic Applications names ten risks.⁴⁸ Map each to the controls in the design, and name an owner for every gap (T5):

Other sources keep the list current: Microsoft’s AI Red Team added seven agentic failure modes in June 2026, among them agentic supply chain compromise, inter-agent trust escalation and MCP and plugin abuse,³⁷ and MITRE ATLAS has added agent-specific techniques since December 2025, such as AI Agent Tool Credential Harvesting.⁴⁹ Tag red-team scenarios with these names so results map to a shared vocabulary. What enters the registry (the supply chain) and what approvers are shown (where trust is exploited) both need named owners, because no gateway closes these risks alone.

Autonomy tiers

Give every agent an autonomy tier and connect each tier to specific controls, monitoring and rollback paths.¹ Approvals at every tier are bound to the exact action and time-limited: if an approval has expired, or the action has changed, by the time it reaches the gateway, the gateway refuses it and asks again.

The tiers are portable. Clients may already classify agents with AWS’s Agentic AI Security Scoping Matrix, BCG’s four autonomy stages or KPMG’s agent types.^33–35 The table shows the closest equivalents, as an approximate mapping by the author.

Tiers move in both directions, and the rules for each move are set in advance and recorded in T5:

• Promote on evidence. An agent moves up one tier only when its evaluations have met the next tier’s thresholds over a sustained period, its red-team findings are closed, its impact assessment is up to date, and its owner and the risk function agree.^33,34
• Demote automatically. An agent drops a tier, or to propose-only, without waiting for a meeting when a security event is detected, its evaluation scores fall below threshold, an approval fails its binding check, the telemetry it relies on goes stale, or it loses its sponsor.³³

Narrowing authority in delegation chains

Multi-agent work turns one request into a chain of hand-offs, and each hop is a chance for authority to widen or for an unclear instruction to pass unchallenged. Research from Google DeepMind on intelligent delegation suggests four rules for every hop.³⁶ Check them for every hand-off in T9:

• Split work into verifiable units. Break the request down, contract first, into tasks whose results can be checked before the next hop relies on them.
• Right-size each unit. Route each task to the least costly model or tool that meets its quality bar.
• Narrow permissions at every hop. A sub-agent gets only the permissions its own task needs, never the delegator’s full set, and never on the strength of a role it claims for itself.³⁷
• Stop and ask. When a request is ambiguous, the agent challenges the delegator or asks a person, as its tier requires, instead of passing the ambiguity down the chain.

In the control plane this means on-behalf-of tokens that are exchanged and narrowed at each hop, and one trace that joins every hop, so a chain can be audited end to end and revoked at any link.

Evaluation as a lifecycle

Write evaluation into the operational requirements (T4) as a lifecycle that runs beyond the release gate: test against success criteria before promotion, score a sample of production traces continuously, and keep a standing red-team suite, tagged to the threat lists above, whose results feed tier decisions. Evaluate the trajectory (which tools the agent chose and in what order) as well as the final answer.⁶⁰ A large public red-teaming competition run with NIST’s Center for AI Standards and Innovation shows why: more than 250,000 attack attempts from over 400 participants found at least one successful hijacking attack against every one of the 13 frontier models tested.⁶¹

Evaluation runs with any evaluation tool. Amazon Bedrock AgentCore Evaluations evaluates sampled live traces online and runs on-demand tests in CI/CD pipelines,⁶² and Microsoft describes ASSERT, an open-source framework that turns behavior specifications into evaluations.⁶³ Whichever tools produce the results, record them in the registry so that promotion decisions rest on them.

Compliance and oversight you can exercise

Record each obligation with the evidence it requires and where that evidence is produced. For high-risk AI systems, the EU AI Act requires that the people overseeing them can understand their limits, override or reverse their output and stop them safely, and that deployers give them the competence, authority and support to do so.³² Measure oversight by that ability. Names on an approval list do not show it, so confirm that approvals are bound to the executed call and expire, that stale data triggers a safe default, and that people have stopped an agent in a drill.

Human takeover after the kill switch

Stopping an agent is only half the job, because someone still has to handle the work it was doing. BCG advises defining clear protocols for people to take over when agents are taken offline, and deciding in advance where agents may not go at all.⁶⁴ Write a takeover runbook for every agent before it reaches tier 3, and record it in T5:

• Stop. The kill switch or credential revocation blocks new actions. Rules set in advance for each tool say whether an action in flight completes, rolls back or is parked.
• Hand over. Open work moves to a named team’s queue with what the agent knew: its trace, pending actions and approvals, and the data it touched.
• Continue. The process runs in a defined degraded mode, manual or deterministic, with its own service levels, until the agent returns.
• Recover. Find the cause, re-evaluate the agent and bring it back one tier at a time.

Then rehearse it, because people tend to trust what AI produces. In a global study of more than 48,000 people, 66% of employees who use AI at work said they rely on its output without evaluating its accuracy, and 56% said they had made mistakes in their work because of AI.⁶⁵ Organizations are adding checks: in KPMG’s first-quarter 2026 pulse, 63% required human validation of agent outputs, up from 22% a year earlier.⁸ A takeover drill tests whether people have the time, information and authority to act, and its timing belongs in the scorecard, under time to contain.

Mapping to the standards auditors use

Compliance and audit teams map every design to the frameworks they report against. The handbook’s Appendix C maps the control plane to ISO/IEC 42001, the AI management-system standard, with ISO/IEC 42005 for impact assessments.⁷² The same appendix maps it to the NIST AI Risk Management Framework⁷³ and to the EU AI Act’s requirements for high-risk systems.³² The summary below is the author’s indicative mapping: confirm clause-level mappings with the client’s compliance team, and add the control IDs to T4 and T5 when they ask for them.

The Cloud Security Alliance’s AI Controls Matrix is also worth knowing: it maps 247 control objectives in 18 domains to ISO/IEC 42001, the NIST AI RMF and the EU AI Act, with guidance for each role from model provider to AI customer.⁷⁴ So is Singapore’s Model AI Governance Framework for Agentic AI, which asks organizations to bound agents’ autonomy and access up front and to define the checkpoints where human approval is required.⁷⁵ The standards connect to the tiers through the impact assessment: an up-to-date one is part of the evidence for promoting an agent.

When agents can reach equipment

In operational technology, the control plane governs what agents may request but is not the last line of defense. Keep agents out of the control network, let agents propose and people with authority approve, limit cumulative change across requests and sessions, and fail to the engineered safe state. The handbook’s Section 5 and reference use case 3 apply these rules in full.