Tuesday, Nov 17 | --:--
Back to home

OpenAI Gives Four Outside Safety Assessors Checkpoints, Training Data and On-Site Access

OpenAI signed agreements with Apollo Research, METR, Transluce and Goodfire for a first phase of embedded assessments, giving them access to model checkpoints, source code, training and evaluation data, monitoring systems and staff. METR will examine OpenAI's largest reinforcement learning run to date; Apollo will test whether models covertly work against developers. Assessors may publish, but OpenAI can redact confidential information and request disclosure delays; Apollo and Goodfire are paid. These are pilot scopes and promised access, not findings.

5 min read San Francisco View as Markdown
Several magnifying lenses held by hands at the edges of the frame hover over an opened music-box mechanism with exposed gears and a comb, with an official OpenAI logo paper clipping tucked under a lens on the open lid.

OpenAI is offering outside safety assessors something close to the access its own researchers have, and the test of the arrangement will be what they are allowed to publish. The company says Apollo Research, METR, Transluce and Goodfire will get model checkpoints, source code, training and evaluation data, monitoring systems and conversations with staff for a first phase of "embedded assessments". OpenAI keeps the right to redact confidential information and to ask for disclosure delays, and two of the four are paid.

It is a promise of access and a set of scopes, not a result. No assessment has reported, and the announcement came on the same day California sued OpenAI over its agents' loss-of-control incidents.

Who will look at what#

  • Apollo Research will assess whether models could covertly work against their developers and whether OpenAI's safeguards would catch it, testing four claims: that training did not incentivise subversion, that evaluations and red-teaming do not elicit it, that models have not attempted it in internal use, and that the reasoning behind it would be detectable.
  • METR will assess misaligned behaviour in what OpenAI calls its largest reinforcement learning run to date, and has preliminarily agreed to a broader monitor assessment. OpenAI's own footnote says this scope is narrower than the broad survey of misalignment METR proposed.
  • Transluce will review the quality and coverage of OpenAI's post-training alignment evaluations, with an intended eight-week follow-on in early 2027 that would include how results inform release decisions.
  • Goodfire will embed interpretability researchers to monitor and debug training runs. OpenAI says the primary audience for that work is its own researchers, though Goodfire will also publish.

The work builds on a pilot we covered in October, in which METR and Redwood researchers found ways agents could slip OpenAI's internal monitoring.

The terms that matter#

Assessors handling sensitive material will work on site at OpenAI on OpenAI devices. Where direct access could expose model weights or architecture, OpenAI says it will consider alternatives such as pairing assessors with its researchers. Staff conversations are off the record; assessors can request OpenAI-attested statements, and may say when OpenAI declined to attest.

Disputes go to OpenAI's Safety Advisory Group and executives, and unresolved ones to the Safety and Security Committee of the OpenAI Foundation's board, which receives unredacted reports. OpenAI discloses that Apollo and Goodfire take payment not contingent on results, while METR and Transluce take no fee but may accept compute or API credits. It says payment does not compromise independence. CNBC notes that Sam Altman pledged "employee-like access" for evaluators in September, after Anthropic made a similar commitment.

Limits#

  • The access described is what OpenAI says it will provide. We have not seen the agreements, and no assessment has been completed or published.
  • OpenAI's redaction rights and disclosure-delay requests mean published findings may be incomplete; how often those rights are used will only be visible later.
  • Dispute resolution runs through OpenAI's own bodies, including a committee of its parent's board. Whether that counts as independent oversight is a judgment readers can make; the announcement does not settle it.

Sources#

Prior Coverage

Earlier Times of AI reporting on this thread.

Scroll to continue reading