Anthropic has let six outside AI testers inside the company, and it can’t stop what they publish

Six people who don’t work for Anthropic now have Anthropic laptops, desks in its offices and roughly the same access to its internal systems as its own risk teams, and the company says it can’t stop them publishing what they find. Anthropic said on Thursday it has started giving “employee-like access” to a team from METR, a nonprofit that tests frontier AI models, which will publish the results of its risk assessments of the company behind Claude.

It is the first concrete step on a promise Dario Amodei made last month, when he called for the industry to “pace the frontier” and said Anthropic would embed outside evaluators without waiting for anyone else. It also answers a pledge in the White House Accord on Super Intelligence, where the big labs agreed to partner with independent auditors or evaluators.

What METR’s team can see

Anthropic says it has onboarded six METR employees with company hardware, office space and access to internal tools “roughly on par” with what its own risk assessment teams use. Staff are encouraged to share information with them as they would with a colleague working on alignment risk.

There are limits. A small number of “clearly defined categories” are off limits, including customer data, legally privileged material and information about Claude’s model architectures. The METR team works under enforceable agreements and is walled off from the rest of METR, including researchers who work with Anthropic on other projects.

METR has worked with Anthropic before: it has reviewed several of the company’s Risk Reports over the past year, and in March one of its staff spent three weeks red-teaming Anthropic’s internal agent monitoring systems and found several new vulnerabilities. It sat alongside Apollo Research when senators heard evidence about OpenAI’s rogue agents on October 1.

“Anthropic can’t veto publication”

The key detail is who controls the findings. METR will publish its analysis of Anthropic’s risk levels, incidents or practices, and Anthropic says it has no veto. It can require redactions only when information falls into the same narrow, pre-agreed categories. METR is free to say publicly that something was redacted and whether it thinks the redaction was justified, as long as it doesn’t reveal what was removed.

Anthropic says it can’t publish everything about its safety work, because some could help attackers, so it is letting outsiders see it instead. It compares the set-up to bank supervision, where regulators place examiners inside the biggest banks. Amodei made the same comparison in his September essay:

Each frontier AI company commits to giving ongoing, employee-like access to a team of embedded third-party evaluators (such as METR), whose role is to verify adherence to safety practices and commitments, report incidents, and help assess the alignment of not just completed AI models but training pipelines and processes.

Dario Amodei, Anthropic CEO, in “We must pace the frontier”

Three tiers of access

METR is the only evaluator with full access so far. Anthropic says it is building three tiers:

• Tier one: narrowly scoped investigations. Anthropic decides what the evaluator sees, and findings are published only with its approval.
• Tier two: more information, and evaluators can publish without approval, subject to the same redaction categories.
• Tier three: the METR arrangement, with employee-like access and the same publication rights as tier two.

SecureBio, a biosecurity nonprofit, is in tier one. It is reviewing the chemical and biological weapons chapter of Anthropic’s August Risk Report, after publishing a pilot review of the previous one. Anthropic is also working with Accenture and Faculty on a broader partnership covering red-teaming, alignment assessments and safeguard testing, and says it is in early talks with others, including Hugging Face.

The catch

All of this is voluntary, and it rests on Anthropic’s own description: METR hasn’t yet published its own account of the arrangement, and there is no date for its first report. The accord behind it is, in President Trump’s words, only “morally binding”. Anthropic says every frontier AI company should do the same. How much outsiders get to see is a live question across the industry: OpenAI let three safety researchers go for sharing information with an outside safety group.

The timing also matters. Anthropic is preparing to go public, and independent reports it can’t veto will now land alongside the risk warnings in its IPO paperwork. They could cut either way for investors.

Why it matters

Until now, outside checks on frontier AI have mostly meant testing a finished model for a few weeks before launch. Anthropic is letting an outside team watch how Claude is built, with the right to say publicly what it finds. The test is whether METR’s first report is genuinely critical, and whether OpenAI, Google and others follow.

Sources: Anthropic, Dario Amodei, METR.