Tuesday, Oct 20 | --:--
Back to home

Anthropic Gives METR Team 'Employee-Like Access' and Says It Can't Veto Publication

Anthropic said on Oct. 15 that six METR staff now have company hardware, office space and internal-tool access roughly on par with its own risk teams. METR will publish its analysis; Anthropic says it can't veto publication but can require redactions in narrow, predefined categories. It also set out three tiers of evaluator access.

Times of AI Desk 3 min read San Francisco, CA View as Markdown
Cover illustration for Anthropic Gives METR Team 'Employee-Like Access' and Says It Can't Veto Publication

Days after Reuters reported labs seeking control over evaluators' findings, Anthropic is describing an arrangement with no publication veto. Anthropic said on Thursday, October 15 that it has started giving "employee-like access" to a team from METR, the nonprofit that evaluates frontier AI models.

What the access covers

Six METR employees have company hardware, office space and access to internal tools and systems "roughly on par with what our own risk assessment teams have access to," Anthropic says. Off limits are a small number of categories such as customer data, attorney-client privileged information and model-architecture information.

METR will publish its analysis of Anthropic's risk levels, incidents or practices. "Anthropic can't veto publication," the company says, but it can require redactions in narrow, predefined categories, and METR may publicly say whether it thinks a redaction was justified. The embedded team works under enforceable agreements and is siloed from the rest of METR. "Our report will openly describe the scope, access, and other limitations of our work," METR said, according to Reuters via CNA.

Three tiers

Anthropic describes three tiers: narrowly scoped investigations published with its approval; publication without approval but with redactable categories; and METR's embedded arrangement. SecureBio is reviewing the chemical and biological weapons chapter of its August Risk Report, and Anthropic says it is working with "Accenture and Faculty" and is in early talks with others including Hugging Face. Anthropic links the move to its signing of what it calls the White House Accord on Super Intelligence, published as the Joint Commitment on Frontier Responsibilities.

Reuters reported on October 13 that Anthropic and OpenAI had sought control over what evaluators publish. A person familiar told Reuters that Anthropic had since become more open in negotiations.

Limits

  • The terms are as Anthropic describes them; the agreements were not published. METR's six staff are not named.

Sources

Prior Coverage

Earlier Times of AI reporting on this thread.

Scroll to continue reading