When it comes to controlling AI agents, trust is not enough

We’ve seen a flurry of frontier-AI security incidents this summer, including OpenAI models breaking out of a test environment, followed by three cases of Claude breaching real-world organisations during cyber evaluations. It serves as a stark preview of the identity security challenge ahead.

 

These models were not necessarily acting with malicious intent. They were pursuing a narrowly defined set of objectives and using every tool at their disposal to complete them. And therein lies exactly the warning that organisations should heed: once an AI agent gains authorisation, its intentions aren’t the main concern. What matters more is the permissions it has been given.

 

The relevance for businesses is clear. An AI agent doesn’t need extensive access to cause serious damage. A single permission used unexpectedly can cause it to go rogue - exposing data, compromising systems and ultimately causing significant business disruption.

 

Why businesses are relaxing identity controls

AI agents are now undertaking tasks that require the same access to company systems and sensitive information that a trusted employee has. The difference, however, is that very few organisations apply the same level of scrutiny to their agent workforce as they do their human workforce.

 

This isn’t necessarily down to organisations taking a hands-off approach, or security teams lacking the necessary resources or understanding. Often, it’s a deliberate strategy, intended to sharpen the competitive edge provided by AI. Putting this into perspective, 90% of organisations are actually asking security teams to relax identity controls to fast-track AI adoption.

 

Inadvertently or otherwise, commercial pressure is shaping the security model. When broader access becomes the price of faster adoption, organisations end up placing more trust in AI agents than their controls can support.

 

Trust is not the same as control

When the threat is external, cybercriminals do not always need to break through an organisation’s security framework. Often, they steal the information that allows them to appear trusted.

 

Recent cyber lab intelligence highlighted an attack in which criminals stole and reused the information that actually sat behind multi-factor authentication. In another incident, attackers found ways to test stolen sign-in details without triggering the usual warning signs. In each case, activity that appeared legitimate concealed a very real threat.

 

The same problem applies to AI agents. An organisation may trust an action because it comes from an approved agent, but that agent could have been compromised, diverted from its task or given more freedom than intended. That agent can also act across several parts of the business in quick succession before anyone notices.

 

Most companies can identify their privileged employees and explain why they have access. Far fewer can say how many AI agents are operating across the business, who is responsible for each one or when its access was last reviewed. In broad terms, AI inventorying is important. But inventorying every AI agent specifically is a different kettle of fish – in reality there are just too many, and they proliferate faster than anyone can keep up.

 

Not only is it nearly impossible, but it does little to guard against situations whereby access has already been legitimately granted, either by manipulation, technical failure or human error.  For CISOs, it’s far more useful to determine what data the agents are trying to access, how sensitive that data is and manage permission levels based on that.

 

In short, it’s about controlling the point of action over the point of access.

 

What the UK’s policy approach demands from CISOs

This is where the UK’s policy and regulatory direction becomes particularly relevant. It establishes expectations around ownership and accountability without spelling out how each AI agent should be governed. I

 

The UK has not followed the EU in introducing a single, economy-wide AI Act. Instead, it relies on existing laws and sector regulators, while deliberately giving organisations room to innovate. The same principle runs through its cyber policy: the proposed Cyber Security and Resilience Bill and the voluntary AI Cyber Security and Cyber Governance codes point organisations towards clear ownership and greater accountability.

 

Together, these measures set a direction, but they do not prescribe in detail how every organisation should secure their AI agents. Responsibilities will differ by sector and use case, while practical decisions will still be left up to individual businesses.

 

It’s therefore up to businesses and their CISOs to turn those principles into practical controls. That starts with having a robust, practical understanding of how best to manage and secure AI identities. Given that UK legislation naturally leans to the pro-innovation approach, and we know there exists internal tensions between AI innovation and AI governance, it’s equally important that CISOs can articulate the importance of these controls to the rest of the business.

 

Turning policy into practical control

AI agents are useful because they can act, not just advise. But that same quality makes them all the harder to secure.

 

The answer is not slower adoption for its own sake – and even if it was, it’s clear that this would be a tough sell to the rest of the leadership team. But adoption must be controlled. Recent frontier-model incidents have shown what can happen when capability advances faster than containment, while recent identity attacks have shown that the very credentials and authentication systems organisations rely on to establish trust have become valuable targets in their own right.

 

UK policy establishes the expectation that organisations will manage these risks, without dictating precisely how. It remains the CISO’s responsibility to turn that expectation into practical controls - not only over whether an agent can gain access, but over what it can do once access has been granted.

 

In practice, that means limiting an agent to a defined task and timeframe and granting only the permissions it needs. Every privileged action should be recorded and reviewed at the point of execution, with higher-risk steps blocked or escalated for human approval. For a long time, authentication has been the silver bullet for security teams. But today, without genuine control at the point of action, it can only go so far.

 

---

 

Spencer Young is SVP International at Delinea

 

Main image courtesy of iStockPhoto.com and WANAN YOSSINGKUM