METR Estimates Anthropic Is Improving Well-Specified Tasks About Twice as Fast, After Anthropic Redacted Most of the Evidence
METR's November 5 public assessment, after an August 12–October 29 high-access review, says Anthropic is likely improving capabilities on well-specified tasks at around twice the March 2024–October 2025 rate, with a subjective range of about 1.3× to 3×. Scores on AECI, Anthropic's version of the Epoch Capabilities Index, grew at an estimated 1.9× the pre-October 2025 rate (95% confidence interval 1.5–2.8×). METR says the acceleration was not from faster training-compute scaling and guesses that about 80% of the current acceleration is due to AI use, mainly assistance rather than fully autonomous R&D. Anthropic redacted a majority of the evidence METR found most informative.
The independent evaluators Anthropic invited in now say the lab's progress on well-specified tasks has roughly doubled — and that the public cannot see most of the evidence behind that conclusion. After a high-access review from August 12 to October 29, METR published a November 5 assessment concluding that, as of October 2026, Anthropic is likely improving model capabilities on well-specified tasks at around twice the rate it did between March 2024 and October 2025. METR says it would not be surprised if the true figure were closer to 1.3× or 3×.
This engagement is not the employee-like access arrangement Anthropic described on October 15. METR says the work started in mid-August, that its staff did not have query access to unreleased models, and that the project is distinct from the assessment announced on October 15. It is another high-access METR review in the same season as the OpenAI agent-monitoring pilot METR published with Redwood, and it comes as METR has also been named in an FTC inquiry into agent risks at Anthropic and OpenAI.
What METR says it measured#
On Anthropic's internal AECI benchmark — the lab's version of the Epoch Capabilities Index — scores have grown at an estimated 1.9× the pre-October 2025 rate since October 2025, with a 95% confidence interval of 1.5× to 2.8×. METR's phrasing is that Anthropic appears to have made roughly two years of historical benchmark progress over the last year. The assessors say they are confident the acceleration was not caused by faster scaling of training compute; further details of that evidence were redacted.
METR guesses that a majority — perhaps about 80% — of the current (October 2026) acceleration is due to AI use, primarily assistance rather than fully autonomous contributions to the most important parts of training. It says other, non-AI factors likely played a large role in late 2025, and that there are "signs of life" for more autonomous contributions soon. Interviews and surveys were "consistent with" AI use causing a doubling in the pace of capabilities progress, METR writes, with a wide range of staff estimates centered around 2×. Informal Slack polls of self-reported speedup reached about 12× in October 2026; METR flags those polls as low-effort and unrepresentative, and notes that individual uplift is not the same thing as organization-wide acceleration.
The 1.9× AECI figure sits above the 1.7× acceleration reported in Anthropic's Haiku 5.5 system card. METR says the main difference is the inclusion of recent internal-only models. Robustness checks that vary the statistical link function produce acceleration estimates from 1.1× to 2.9×; other checks that vary snapshots, benchmarks or regularization stay closer to 2×. METR says it did not identify clear bottlenecks in algorithms, training compute, experiment compute or data that would bind in the next 12 months, but that its view of those bottlenecks changed several times as more detail arrived, so it is not confident in that assessment.
What the public cannot see#
Anthropic had unilateral rights to redact non-public information. METR says that while it could share its highest-level conclusions about risk, Anthropic redacted a majority of the evidence the assessors found most informative. Anthropic described the redactions as aimed at removing confidential information that could accelerate other AI developers. The public report is a synthesis of a roughly 100-page full report delivered to Anthropic on October 22; METR submitted the final public version on November 3.
Because much of the redacted evidence had multiple plausible interpretations, METR writes that the report "does not achieve 'transparency about evidence'" and that the public has to rely on METR's judgment more than would be ideal. Detailed estimates of how much of the acceleration to attribute to AI in each part of the training pipeline were among the material withheld.
A compute-allocation check that had been in the original contract — verifying how much compute went to capabilities versus safety — was dropped mid-engagement, METR says, before it reached substantive conclusions. Anthropic said it wanted the work focused on risk assessment and would revisit compute verification later.
Limits#
- The ~2× conclusion is METR's subjective synthesis, offered with high uncertainty and internal disagreement on the team.
- Most of the evidence METR says most informed that view is redacted; readers cannot inspect it.
- AECI measures performance on well-specified, automatically gradable tasks. METR warns that acceleration on those tasks may not map cleanly to messier real-world work.
- Staff self-reports of 12× uplift are, in METR's own words, only very loose evidence and do not equal an organization-wide acceleration rate.
- METR did not sample unreleased models directly and did not have employee-like, unmediated access to Anthropic systems.
- We have not independently recomputed the AECI fits.