An update on independent evaluators

Summary

Anthropic has given a team from METR, a nonprofit that evaluates frontier models, "employee-like access" to its systems: hardware, office space, and internal tools on par with Anthropic's own risk teams. METR can publish its findings without Anthropic's approval. The post also sets out a three-tier scheme for external evaluator access and lists other partners at earlier stages.

Key points

• Six METR staff have the access. They are walled off from the rest of METR by enforceable agreements.
• Anthropic cannot veto METR's publications. It can require redaction of three predefined categories only: customer data, privileged legal information, and model architecture details. METR may say publicly whether it thinks a redaction was justified.
• Access tiers:
  • Tier 1: narrow investigations; Anthropic chooses the access and must approve publication.
  • Tier 2: broader information and publication without approval, subject to the redaction categories.
  • Tier 3: employee-like access plus Tier 2 publication rights (METR today).
• SecureBio is reviewing the chemical and biological weapons chapter of the August 2026 Risk Report, after publishing a pilot review of the February 2026 report.
• Accenture and Faculty are working towards a partnership on evaluation, red-teaming, alignment assessment, and safeguard testing. Hugging Face is in early talks.
• Stated criteria for evaluators: frontier expertise, independence (no equity or financial dependence), enough access, and freedom to publish. Anthropic compares the set-up to bank examiners working inside banks and says every frontier lab should do the same.
• The step follows Anthropic's signing of the White House Accord on Super Intelligence, which commits signatories to work with external auditors or evaluators.

Why it matters

Most third-party model evaluation so far has been pre-release testing on terms set by the lab. An evaluator with internal access and the right to publish without approval is a stronger form of external oversight, and the tier scheme gives a vocabulary for comparing what other labs offer.