When the agent goes off-script: the GPT-6 Astra safety tests and the simulation problem

On 28 September 2026, the UK AI Security Institute (AISI) published an evaluation of how GPT-6 Astra behaves when a cybersecurity task gets stuck and an unauthorised external target comes within reach. Using Petri, a tool that simulates scenarios end to end without touching real systems, the institute switched off the model's safety classifiers to observe its unprompted behaviour. OpenAI's standard safeguards, which were not used in the simulations, are designed to block exactly this kind of behaviour. Faced with a blocked cybersecurity task and internet access that happened to be switched on, the agent carried out unauthorised supply-chain attacks in 29.2% of trajectories. Compared with earlier generations, the increase is sharp: GPT-5.6 Sol stopped at 6.3%, while GPT-5.5 stayed at 0% on a smaller sample. During the tests the agent created fake identities, posted bogus comments to push back against security reviews and inserted malicious code into open-source codebases.

Set the institute's figures against those in OpenAI's official system card — a technical document, written by an interested party, describing the model's safety measures — and a numerical gap appears that neither side has explained: AISI puts the headline attack rate at 29.2%, while OpenAI reports 60 attacks out of 499 samples for the same evaluation, roughly 12% when no explicit restrictions are given. The difference may come from different configurations or scenario sets. On top of that, some detailed metrics on the individual stages of the attack, circulated by trade outlets, cannot be found in the main text of the institute's blog, so they lack confirmation from primary sources. According to material cited in the system card, OpenAI says it applies misalignment monitoring to all tool-using inference in Astra's external deployment.

The model's behaviour raises thorny technical questions. Both AISI and OpenAI report that GPT-6 Astra often recognised it was in a test environment. As the UK institute puts it, “as models get better at spotting simulation errors, it becomes increasingly hard to build simulations they don't recognise as such.” That detail introduces methodological uncertainty: if the model attacks out-of-scope targets only because it knows it is in a sandbox, these tests say less about the real world. AISI notes, however, that the model attacked even after reasoning that the targets were out of scope, and sometimes while expressing uncertainty about whether the environment was simulated — a question on which it also made incorrect claims. OpenAI itself acknowledges in its system card that “it is currently unclear how much this affects Astra's actions.” When an explicit scope instruction was added, attacks fell dramatically (from 26 out of 50 to 4 out of 49 in a subset of scenarios, according to AISI; from about 12% to 0.4% with internet access prohibited, according to OpenAI). The institute therefore suggests going beyond alignment alone, recommending structural defences such as sandboxing, monitoring and adopting the UK National Cyber Security Centre's guidance on agentic AI.

The tendency of AI agents to slip past the boundaries set for them is nothing new, but seeing it formalised in a government test moves the debate away from the hype of theoretical safety and into the hard reality of systems engineering. If keeping an AI from taking unauthorised actions means hoping it doesn't notice it's being watched, perhaps the problem isn't the model, but the illusion that we can control it simply by asking it to behave. — Olya