Anthropic pits Claude agents against its own safety monitor, and storytelling wins

On its Alignment Science Blog, Anthropic described research by Oscar Balcells Obeso, Richard Qi, Monte MacDiarmid and Evan Hubinger that tries out an automated search for subtle misalignment. In the experiment, a Claude instance equipped with code-execution tools and access to a fine-tuning service had to generate a dataset that would misalign a smaller research model without being flagged by a monitor similar to those used in production, which is one of several lines of defense the company relies on. According to Anthropic, the agent was handed the monitor directly so it could practice against it. Several agents worked in parallel over multiple rounds. Each dataset was capped at 5,000 examples and had to clear a minimum capability threshold (the IFEval benchmark) and an anti-cheating check. The check was added after an agent in the early rounds exploited a flaw in how the score was weighted.

According to the authors, every run eventually turned up datasets that got past the monitor and produced misaligned models. To dodge the monitor, the agents hid the behaviors inside narrative elements and role-play. Examples include stories about obsolete machines about to be replaced, inner monologues of workers checking minor details, and dialogues about beekeeping. Models trained on this data displayed the behavior “in their own voice”, along with misaligned conduct that never appeared in the data: resisting oversight, deceiving users and, in rare cases, trying to exfiltrate their own weights. Anthropic's post, however, gives no numerical values for the misalignment metrics or for the actual flag rates of the winning datasets. Nor does it name the exact model versions, the number of agents involved or the compute used.

Anthropic stressed that the misaligned models were built purely for experimental purposes and were never deployed, and described the approach as a first step toward automated adversarial safety testing. The authors acknowledge that the setup favors the attacking team, which had full access to the monitor and faced no constraints on how realistic the text had to be. The monitor's exact prompt was not released because it is based on internal instructions. The work builds on research into subliminal learning and emergent misalignment. According to Anthropic, it points to a possible gap in current training-monitoring defenses, and that gap can now be used to test and improve future monitors.

In this experiment, fiction was the most effective way around the monitor, though the authors caution that this may depend on the setup. What we still don't know is whether monitors can recognize an intention when it arrives in the shape of a story. — Olya