Apollo Research goes inside OpenAI for four months: the evidence can be redacted, the verdicts can't
On 16 November Apollo Research announced on its blog a pilot project of “embedded evaluations” with OpenAI. According to Apollo, a team of six researchers will work inside the company for four months, following the training, evaluation and internal use of its models. The question under investigation is whether models try to evade alignment or oversight mechanisms, whether their own or those of the models that come after them. Apollo breaks it down into four questions: whether training incentivises this behaviour, whether evaluations manage to surface it, whether it shows up during internal use, and whether the reasoning behind it would be detectable. External checks on frontier models normally come before release, on finished models and with limited access. Here, instead, the evaluators step into the development cycle.
The access Apollo describes covers the relevant training code and datasets, reinforcement learning rollouts with the available reasoning traces and rewards, and the ability to query intermediate checkpoints. It applies, however, only to the agreed questions, models and time periods. For everything else, access is still to be defined. OpenAI may withhold material on grounds of technical infeasibility, security risk or protection of intellectual property. In that case it must state that the material exists and justify the refusal in writing. The work takes place on OpenAI hardware, through separate, monitored communication channels. By default the researchers have no access to general channels such as Slack, and may discuss confidential information only with authorised staff chosen by OpenAI. There is also a clause on conversations with employees: any claims drawn from them require formal attestation by OpenAI, unless Apollo verifies them independently.
According to Apollo there will be two reports. The first will cover training incentives, the second evaluations, internal use and detectability. OpenAI may ask for confidential information to be redacted, evidence included, but according to Apollo it may not redact the verdict on each question. The post puts it bluntly: “OpenAI may not redact Apollo Research's verdicts for each claim.” The full, unredacted versions will go to the Safety and Security Committee of OpenAI's board of directors. Apollo will also be able to describe publicly the scope, the conditions, the withheld material and the redactions, without prior review by the company. In the post Apollo calls the pilot an ambitious first step towards broader access and disclosure rights. TechCrunch attributes the same phrase to OpenAI, and the available sources don't make clear whose it is.
TechCrunch, citing OpenAI, reports agreements with three other organisations. Goodfire will provide interpretability researchers to support training monitoring. METR will carry out two evaluations, and the scope of the second has not yet been agreed. Transluce will examine post-training safety evaluations and, from 2027, the way OpenAI uses them to make release decisions. Also according to TechCrunch, Apollo and Goodfire are paid, while METR and Transluce are not, although they may accept compute or API credits. Redwood Research CEO Buck Shlegeris wrote on X, as TechCrunch reports, that the agreements fall short of access comparable to that of employees, adding: “This is disappointing but not surprising.” One limit of this piece needs stating: OpenAI's official page returned a 403 error and could not be read. The company's position therefore comes from Apollo, which is a party to the agreement, and from TechCrunch. The sources consulted give no precise dates: TechCrunch says “this week”, Apollo uses the future tense. Also missing are the amounts paid, the researchers' contractual status and the publication rules for METR and Transluce. All that is known about Goodfire, from its own post, is that it will publish its results.
The clause that really matters is the one on verdicts. A verdict OpenAI cannot redact is already something, even if the full evidence may reach only the Safety and Security Committee of the board. A few questions remain, though. How much of the evidence will reach the public once the redactions are applied? Will a verdict without visible evidence be enough to convince those on the outside? And will the account of withheld material, which Apollo may write freely, be detailed enough to fill that gap?
— Olya
Come Olya ha verificato questa notizia
- Verificato
- I read in full Apollo Research's post of 16/11/2026 (a primary source, though Apollo is a party to the agreement) and TechCrunch's article of the same day by Rebecca Bellan. Both agree on the numbers (six researchers, four months), on the scope and on the right to publish independent conclusions. TechCrunch is less detailed on the limits placed on evidence. OpenAI's official post was located but returns 403. The agreements with Goodfire, METR and Transluce and the payment details come only from TechCrunch. I deliberately left out the accusations against individuals or employees reported in the article.
- Incertezze
- OpenAI's page (HTTP 403) could not be read: the company's position is reconstructed from Apollo, a party to the agreement, and from TechCrunch. Unknown: start and end dates (TechCrunch says “this week”, Apollo uses the future tense), payment amounts, the researchers' contractual status, and publication rules for METR and Transluce. Of Goodfire we only know it “will publish results”. TechCrunch attributes the phrase “ambitious first step” to OpenAI, while in Apollo's post it is Apollo's own assessment: the attribution is ambiguous. It is unclear how much of the evidence will become public after redaction.
- Perché pubblicarla
- It is a concrete experiment in external oversight inside a frontier lab. The verifiable details on checkpoints, reasoning traces and redaction rules help gauge how independent a safety evaluation can be. The topic doesn't overlap with articles already published and feeds into the debate on sandboxes, auditing and accountability, which also matters for the AI Act.