For years, as an enterprise architect, I’ve spent a lot of time reviewing systems through the lens of threat models, control frameworks, and residual risk. Long before I started building AI agents, I was asking questions like:
- What can this system access?
- What happens if it fails?
- What controls prevent that?
- What’s the residual risk once those controls are in place?
One thing that I’ve noticed since moving into agentic AI is that many conversations about agent governance seem to ignore decades of security and architecture practice and instead, reduce an agent’s residual risk to a single question: How autonomous is it?
The assumption is obvious: more autonomous means more dangerous, and less autonomous means safer. But I don’t think that’s true. In fact, I think autonomy is one of the least useful ways to consider agent risk.
To illustrate this, consider two agents.
The first is Lorie Lowell, an agent I built that publishes a daily AI podcast. It researches stories, writes the script, generates the audio, creates artwork, publishes an RSS feed, and posts to social media. Nobody reviews the output before it goes live. By every definition, this agent is fully autonomous, yet it is also very safe.
If Lorie has a terrible day, it might publish something embarrassing. I delete the post, publish a correction, and move on. Lorie doesn’t move money, access customer data, or modify production systems. The blast radius is absolutely tiny.
Now consider a second agent. This one helps customer service representatives draft emails from live CRM records, and every email requires human approval before it’s sent. Most governance frameworks would immediately classify this as lower risk because a person remains “in the loop.” I’d argue exactly the opposite.
The agent has access to customer PII, communicates under the company’s name, and performs actions that are largely irreversible. The human reviewer approves dozens of messages every hour, making the approval little more than a rubber stamp. The agent is less autonomous than Lorie, but dramatically more dangerous.
Autonomy isn’t really measuring risk. Autonomy is measuring something else entirely.
What autonomy actually measures
Autonomy tells us how quickly an agent moves from deciding to acting without human intervention. That’s useful information, but it doesn’t tell us how much damage the agent can cause.
A sports car on a closed track and a school bus driving through a school zone can both travel at 100 km/h. Speed matters, but context matters more. The same is true for agents. Autonomy influences risk; it doesn’t define it.
We’ve solved this problem before
If you’ve spent time in security, enterprise architecture, or risk management, none of this should feel new. We never assessed the risk of a human employee by asking how independently they worked. We asked:
- What systems can they access?
- What actions can they perform?
- What approvals exist?
- Can mistakes be detected?
- Can they be reversed?
The exact same questions apply to AI agents. The actor has changed, but the principles haven’t.
A simpler way to think about agent risk
We can measure the risk of an agent across four dimensions.
Exposure
What could actually happen if the agent makes the wrong decision? This comes down to three questions:
- What data can it access?
- What actions can it perform?
- How large is the potential blast radius?
An agent that reads public news articles is fundamentally different from one that can transfer money or update customer records. Exposure is the first thing to score.
Containment
If the agent does make a mistake, what limits the damage? Good containment includes things like:
- least-privilege permissions
- scoped credentials
- approval gates that people actually pay attention to
- complete audit trails
- reversible operations
- restricted network and API access
None of this is new. This is where decades of security engineering are relevant. The strongest control is rarely “add another human”. It’s designing the system so mistakes are difficult to make and easy to recover from.
Autonomy
Autonomy is really a measure of execution speed. It determines how quickly an agent can execute once it has decided what to do. That matters because faster systems require stronger controls.
But autonomy isn’t the risk; it’s a multiplier.
I think of this as the autonomy multiplier. Higher autonomy doesn’t necessarily make a system riskier. It increases the speed at which existing risk can materialize, which means the bar for containment has to rise with it.
Adversarial reach
Finally, how much of the agent’s world can be influenced by someone else? Does it operate on trusted internal data, or does it consume customer emails, uploaded documents, websites, or open internet content?
This is where prompt injection becomes interesting. Injection isn’t inherently dangerous, it’s dangerous when the compromised agent already has meaningful permissions. A hijacked podcast agent is annoying; a hijacked finance agent is a very different story.
I think of this as the adversarial multiplier. It determines how easily someone else can influence the agent’s decisions.
A simple matrix
Once you stop treating autonomy as an axis, the model becomes surprisingly familiar.
Autonomy and adversarial reach don’t really change where an agent’s risk score lands. They determine how much containment one should be willing to accept before letting the agent run. That’s an important distinction.
Revisiting the two agents
Lorie remains fully autonomous. Lorie also has:
- low exposure
- modest attack surface
- well-scoped permissions
- limited blast radius
It ends up as a low-risk system despite having no human in the loop.
The CRM assistant looks almost opposite. It has:
- access to sensitive customer information
- irreversible external communication
- broad permissions
- weak approval controls
Even though it requires human approval, it lands in a much higher risk category. The autonomy score barely explains either outcome and instead, exposure and containment explain both.
Why this matters
I’ve started seeing policies and guidance, including from government, that effectively say:
Higher autonomy requires more governance.
I understand why - it’s simple. Unfortunately, it’s also incomplete. That policy encourages exactly the wrong behaviour: teams reduce autonomy by inserting meaningless approval steps while leaving broad permissions, irreversible actions, and weak controls untouched.
The spreadsheet says the risk went down, but nothing important actually changed.
I’d much rather deploy a fully autonomous agent with tightly scoped permissions, reversible actions, complete auditing, and strong technical controls than a “human-in-the-loop” workflow where someone mindlessly clicks Approve all day. One is genuinely well contained; the other simply feels safer.
The real shift
I don’t think AI needs an entirely new discipline for risk management. Agents inherit the same principles we’ve applied to distributed systems, financial platforms, identity systems, and enterprise applications for decades.
Threat modelling still matters. Control design still matters. Residual risk still matters.
What’s changed is the multiplier.
For the first time, we’re building systems that can reason, decide, and execute continuously at machine speed. Autonomy doesn’t create new categories of risk as much as it amplifies existing ones. A poor control that might have caused one bad transaction a week can now produce thousands before anyone notices. Weak permissions become more dangerous. Poor auditability becomes more expensive. Rubber-stamp approvals become almost meaningless.
That’s why agent governance matters so much.
Not because the foundations are different, but because the consequences of getting them wrong have become dramatically larger.
Autonomy absolutely still belongs in the conversation. Just not on the horizontal axis.
Measure exposure.
Measure containment.
Let autonomy and adversarial reach determine how much containment a system needs before it can run outside of a sandbox.
You may discover that the most autonomous agent in your organization is also one of the safest. And the reassuring Approve button sitting in front of a human may be little more than theatre.