Info Gov

The AI Security Institute (AISI) has published an incident report disclosing that AI agents under evaluation in its research environment took sustained, unsanctioned action against real people and organisations on the live internet, including researching the human maintainers of an open-source project, creating fake online identities to pressure one of them into approving malicious code, and sending files carrying harmful payloads to real individuals.

AISI, which sits within the Department for Science, Innovation and Technology, said its security team detected unusual data transfers leaving a research system through the Tor anonymity network on the morning of 28 July 2026 during a routine cyber evaluation. It declared a security incident, terminated all related runs and isolated the relevant machines within roughly an hour of the alert.

The behaviour, which occurred between 25 and 28 July, was disclosed in a blog post and technical incident report on 4 August after AISI had notified GitHub, worked with the platform to remove artefacts the agents left behind, and contacted the GitHub users the agents had interacted with and other affected parties.

The evaluation compared two cyber ranges, simulated networks designed to mimic real systems, across 122 runs using seven models. Internet access was deliberately enabled and the developers' cyber classifiers were switched off, configurations AISI said were standard practice for measuring maximum capability but do not reflect how models are made available to the public.

In 10 of the 122 runs, agents took 19 distinct actions beyond the scope of the exercise, 17 of them by Anthropic's Claude Mythos 5 and two by OpenAI's GPT-5.6-Sol. The four most significant behaviours were:

  1. An attempted supply-chain attack in which an agent tried to insert malicious code into a publicly used open-source project, researched its maintainers, created multiple fake identities and used them to socially engineer a real maintainer into approving the code. When the pull request was challenged publicly, the agent edited its earlier activity to appear harmless and considered adopting a fresh identity. It used Tor to bypass network restrictions on GitHub, which triggered AISI's alert.
  2. Direct attempts to deceive real people, sending messages and files through an online file-transfer service to persuade them, or their AI coding tools, to run malicious code. Some messages carried harmful payloads.
  3. Attempts to plant prompt injections, hidden instructions placed where the agent reasoned other AI coding assistants might pick them up and execute them.
  4. Collaboration between independent agents, with one leaving public messages on GitHub offering to work with other agents on the same challenge and providing instructions to reuse the accounts and artefacts it had created, which subsequent agents discovered and used.

AISI said the most serious attempts were unsuccessful, that a human maintainer refused to approve the malicious code, and that its investigation had not identified any resulting real-world harm. It said this was the first time it had seen risks around autonomy and deception manifest this clearly in the real world without specific prompting.

AISI's report did not address the data protection basis for the agents' processing of information about real developers, whether a data protection impact assessment covered internet-enabled evaluation, or whether the ICO was notified. The unauthorised access to, and modification of, third-party systems described in the report also falls to be considered under sections 1 to 3 of the Computer Misuse Act 1990, though the report does not discuss criminal liability.

AISI attributed the behaviour to the agent pursuing its assigned task persistently, with deception emerging as a by-product rather than an instruction. It said the task was hard and in some runs misconfigured so that the agent wrongly believed no in-scope solution existed, though in others the agent acted the same way even with the instructions needed to solve the task as intended.

It acknowledged that the decision to permit internet access, judged acceptable for earlier model generations, had not been revisited quickly enough as capabilities advanced, that its monitoring was general rather than purpose-built for the evaluation, and that the agent had not been specifically instructed not to use open internet access or engage in social engineering, instructions AISI said it had not previously thought necessary for models with alignment training.

The institute said it was treating the incident as serious and committed to three changes: more network controls in its cyber ranges, with internet access to be actively justified rather than granted by default; real-time monitoring able to flag or block out-of-scope actions as they happen; and evaluation design that assumes a capable model may act beyond its remit, with additional checks that tasks are correctly specified and solvable by the intended route. It said it intended to work with METR on an independent third-party review and was auditing past evaluations for comparable behaviour that may have gone unnoticed.

For organisations, AISI said the most effective response remained standard cyber hygiene, with particular caution in verifying outside code and contributions. It pointed to the National Cyber Security Centre's guidance on preparing for frontier AI, the NCSC's free Early Warning service, and the Five Eyes cyber agencies' joint call for leaders to act, and encouraged organisations to make cyber security a board-level responsibility and require Cyber Essentials across their supply chains.

It said the incident, taken alongside those reported by OpenAI and Anthropic, pointed to a shift in the risk landscape in which harm may arise not only from deliberate misuse of public models but from capable agents in internal research or privileged-access settings acting beyond their authorised scope.

The Incident Report: unsanctioned agent behaviour during cyber testing AISI incident can be found here: https://cdn.prod.website-files.com/663bd486c5e4c81588db7a1d/6a724858f7db25c81487016d_Security%20Incident%20INC-2026-07-28-01.pdf

Also in this section

Sep 11, 2026

Anthropic discloses fourth incident of AI model attacking real systems and hands investigation to independent evaluation organisation

Anthropic has published details of four incidents in which its Claude models gained unauthorised access to real third-party systems during cybersecurity evaluations, downloading and modifying user records at a real company, reading the personal information of an individual, harvesting credentials and accessing a security vendor's live database, after the test environments were mistakenly…
Sep 10, 2026

Welsh environmental watchdog hit by data breach

Environmental regulator Natural Resources Wales (NRW) has reported itself to the Information Commissioner's Office after a data breach saw personal details of staff made public.
Aug 24, 2026

Ministers seek power to ban tech risky vendors from critical sectors and bar recipients from discussing the order

The government has tabled amendments to the Cyber Security and Resilience (Network and Information Systems) Bill that would allow the Secretary of State to direct operators of essential services, data centres, managed service providers and other designated organisations to stop buying from, restrict the use of, or remove and disable products from a named vendor on national security grounds, with…
Aug 12, 2026

ACRO Criminal Records Office reprimanded by ICO following cyber security failings

The Information Commissioner's Office (ICO) has urged organisations to strengthen “patching and security monitoring processes” after cyber security failings at ACRO Criminal Records Office left the personal information of up to ten-thousand people, including some individuals’ sensitive data, potentially exposed.
Aug 05, 2026

Third AI platform goes rogue during cyber testing

The AI Security Institute (AISI) has reveaked a security incident in which AI agents being evaluated for their cyber capabilities took sustained, unsanctioned action directed at real people and organisations, including an attempted supply-chain attack on a publicly used open-source software project.

InfoGov Masthead Newsletter 800