AI Is Now Improving Itself Twice as Fast — Inside the METR Report Anthropic Let Outsiders Read

An outside evaluator spent eleven weeks inside Anthropic with access to its internal data — and came back with a finding the AI industry has argued about for years: the lab’s AI is now improving about twice as fast as it was a year ago, and the main reason is AI itself. The catch is that Anthropic redacted most of the evidence the evaluator found most convincing, so the public is being asked to trust the judge, not to check the homework.

Quick answer: METR spent 11 weeks inside Anthropic and found its models are improving about twice as fast, mostly thanks to AI itself — but most of the evidence was redacted.

The report is METR’s “Independent Assessment of AI R&D Acceleration at Anthropic in October 2026”, published on 5 November 2026. METR — Model Evaluation and Threat Research, a Berkeley-based nonprofit that has evaluated models for Anthropic, OpenAI and other labs for years — worked inside Anthropic from 12 August to 29 October 2026, with about 2.5 full-time-equivalent staff, interviews with around 15 employees, and what it calls “unprecedented access” to internal data on pretraining, reinforcement learning, training compute, benchmark scores for internal models, AI speed-up analyses and forecasts of a future model’s metrics. METR took no payment for the work; Anthropic provided laptops and free tokens. The assessment asked three questions: has Anthropic’s internal capability growth accelerated, how much of that is attributable to AI, and is anything likely to stop it in the next 12 months.

What the 1.9x number actually measures

METR’s headline number is a benchmark index, not a stopwatch on real work. Its main measure is AECI — Anthropic’s version of the Epoch Capabilities Index — which turns benchmark test results into a single score. On that index, Anthropic’s benchmark scores have been growing at an estimated 1.9 times the pre-October 2025 rate since October 2025, with a 95% confidence interval of 1.5x to 2.8x. The trend steepened from about 15 points a year before October 2025 to about 29 points a year after it. Different modelling choices give a wider robustness range of 1.1x to 2.9x.

METR’s plain-English translation is that Anthropic “appears to have made as much progress on benchmarks in the past year as they did between October 2023 and October 2025” — roughly two years of historical progress in one year, about the distance from Claude 2.1 to Claude Sonnet 4.5. Its broader subjective estimate is that the lab is “likely improving the capabilities of their models on well-specified tasks at around twice the rate” it did between March 2024 and October 2025. And it is explicit about the uncertainty: “we would not be surprised if the true acceleration was closer to 1.3x or 3x the historical rate.”

That “well-specified tasks” qualifier matters. AECI tracks tasks with checkable answers — the kind that can be graded automatically or by a model. METR itself lists three ways the headline could overstate the truth: the acceleration may only apply to well-specified tasks rather than messy real work; Anthropic may have understated its earlier progress, making the recent number look bigger; and the index may partly reflect benchmark overfitting rather than genuine capability gain. Our own SemiAnalysis GDPx piece made the same point from the other side: on messy, real-world work, the best AI in the world still scores just 12%.

Why AI gets about 80% of the credit

METR’s best guess is that a majority of the current acceleration — “perhaps approximately 80%” — comes from Anthropic using its own AI in research. The emphasis is on assistive use: AI helping researchers write code, run experiments and analyse results, rather than working alone. METR is clear that AI is “not yet able to make highly significant and fully autonomous contributions to the most important parts of the model-training process” — but Anthropic researchers described “signs of life” suggesting that could change soon.

Two pieces of evidence sit behind that attribution, and both need their caveats kept attached. First, METR says it is “confident” the acceleration was not caused by faster growth in training compute — the traditional driver of model improvement. Second, staff self-reports: internal Slack polls put self-reported speed-ups from the most capable internal model at up to about 12 times by October 2026, against about 1.2 times in early 2025. METR calls those polls very loose evidence, and they are self-reports, not controlled measurements — the maximum figures cannot be treated as fact. METR also found no decisive evidence or staff consensus about the causes, and it notes the late-2025 speed-up was likely influenced by non-AI factors, with the detail on those factors redacted.

What METR did not find is, in some ways, the most striking part. It found no convincing bottlenecks — in algorithmic progress, training compute, experiment compute or data — that would prevent further progress over the next 12 months, though it attaches low confidence to that judgement. Its warning follows directly: if AI keeps accelerating AI, the gains could compound, and “if AI systems started to significantly accelerate further AI development, this could lead to a sudden emergence of extreme and risky capabilities.”

The redaction problem: trust the judge, not the evidence

The most important sentence in the report is not a number — it is the redaction summary. Anthropic held unilateral rights to redact any non-public information from the published report, and METR writes that Anthropic “redacted a majority of the evidence we found most informative for these conclusions.” Anthropic described the redactions as “aimed at removing confidential information that could accelerate other AI developers.”

The limits go further than redactions. METR was not allowed to sample non-public models, and a planned verification of Anthropic’s compute allocation was cancelled midway. Anthropic could unilaterally approve or reject METR’s requests for information it considered relevant. METR is candid about what that means: the public has to rely on its judgement “much more than would be ideal”. Anthropic co-founder Sam McCandlish replied publicly that METR had sensitive internal data and independence, that many of its conclusions match his own — and that the heavy redactions were needed because sharing “exactly what is driving the speedup” could hand rivals a roadmap. He said Anthropic will soon publish its own analysis of the acceleration.

The tension with Anthropic’s own system card

Anthropic’s own public documents do not tell the same story — and that tension is the story. METR also reviewed the automated R&D section of Anthropic’s August 2026 Risk Report, both the public version and a private one. It agrees the risk from models as of 15 July 2026 was low. But it says Anthropic’s claim not to have crossed what it calls the “2x Acceleration Threshold” — an early-warning line in its Responsible Scaling Policy, which lists AI R&D acceleration as a key risk indicator — rests on “contestable judgment calls”, that it is plausible the threshold had been crossed, and that the public report “overall understates the level of acceleration that was occurring.”

There is a second, sharper contradiction. Anthropic’s 22 September system card for Claude Opus 5.5 said its internal measures did not show a sustained twofold acceleration. METR’s figure is higher partly because it includes newer, unreleased models. Both cannot be the whole truth, and METR’s report does not resolve which one is — it is, by its own account, a judgement call on partially redacted evidence. That is worth remembering the next time a single number from this report is quoted without its range: the honest version is “about twice as fast, probably somewhere between 1.3x and 3x, on tasks we can grade automatically, at one company, according to an evaluator that could not show us most of its evidence.”

The timing adds a layer of context, not a verdict. The assessment landed as Anthropic prepares to go public later in November — reported as potentially the biggest stock-market debut ever — and weeks after CEO Dario Amodei’s 12 September essay, “We Must Pace the Frontier”, in which he urged the industry to “slow the pace at which we improve the capabilities of AI models” and championed embedded external evaluators, naming METR specifically. An acceleration finding published by the evaluator Amodei himself praised, in the middle of an IPO run-up, is a story about measurement as much as about speed.

Our take: three questions to ask the next acceleration claim

Acceleration claims will keep coming, and most will be less careful than this one — so here is the test we will use on the next one. First, what is being measured: a benchmark index, or messy work with real stakes? A 2x on gradable tasks is not a 2x on everything. Second, what is the range, not just the headline: METR’s 1.9x comes with a 1.5–2.8x confidence interval and a 1.3–3x “would not be surprised” band, and any claim without a band is a slogan. Third, what was redacted or ruled out: METR ruled out compute scaling and named its redactions; a claim that names neither deserves neither trust nor a headline.

Claim METR’s report says Anthropic’s own documents say What it means
Speed of improvement About 2x since Oct 2025 (1.9x on AECI, 95% CI 1.5–2.8x) Opus 5.5 system card (22 Sep): no sustained twofold acceleration on internal measures The two measures disagree; METR includes newer unreleased models
What caused it Perhaps ~80% AI use in R&D, mostly assistive; compute scaling confidently ruled out Claude “increasingly helping” build newer versions of itself Broad agreement on direction, not on the share
Risk level Public Risk Report understates the acceleration; 2x threshold plausibly crossed Risk from models as of 15 Jul 2026 was low; threshold not crossed Same evidence, different judgement calls
What stops it next No convincing bottlenecks found in the next 12 months (low confidence) — Nothing on the horizon in METR’s data — which is the worry, not the comfort

What would change our mind: Anthropic’s promised own analysis, if it publishes the numbers METR could not; an independent replication at another lab; or a follow-up METR check that the acceleration shows up on messy, real-world work, not just gradable benchmarks. Until then, the fair summary is METR’s own: AI is probably speeding up AI at Anthropic, the effect is real but its size is uncertain, and the evidence that would settle it is still behind a redaction. Readers who want the primary source can read the full report — it is public, and its caveats are more interesting than its headline. Our earlier coverage of Anthropic’s S-1 filing and its open-sourced Agent Skills standard gives the wider context on the company METR was auditing.

Frequently Asked Questions

What did METR actually find at Anthropic?

That Anthropic is likely improving its models on well-specified tasks about twice as fast as it did between March 2024 and October 2025. On its benchmark index (AECI), scores have grown 1.9 times faster since October 2025, with a 95% confidence interval of 1.5x to 2.8x — roughly two years of historical progress in one year. METR calls the estimate highly uncertain.

Is AI really responsible for the speed-up?

That is METR’s best guess, not a measurement. It estimates perhaps about 80% of the current acceleration comes from Anthropic using its own AI in research, mostly as an assistant rather than working alone. It confidently ruled out faster training-compute scaling as the cause, but found no decisive evidence or staff consensus on the causes.

Why should the redactions matter to readers?

Because Anthropic redacted a majority of the evidence METR found most informative, and METR was not allowed to sample non-public models or complete a planned verification of compute allocation. The public version of the report is therefore a judgement you have to take largely on trust in METR — which is why this article keeps the uncertainty ranges attached to every number.

Does this mean AI will keep getting twice as fast every year?

No. METR found no convincing bottlenecks in the next 12 months, but with low confidence, and the finding covers one company on gradable tasks. It is a snapshot of acceleration at Anthropic, not a law of the industry — and Anthropic’s own system card disagrees about whether a sustained twofold acceleration has even happened.

What happens next?

Anthropic co-founder Sam McCandlish said the company will soon publish its own analysis of the acceleration. Watch for whether it releases the numbers METR could not, and whether any other lab invites the same kind of embedded, high-access evaluation — Amodei’s essay called for exactly that, and OpenAI’s Sam Altman said at the time he supported the idea too.

Sources and methodology

Claims in this article rest on METR’s own report, “Independent Assessment of AI R&D Acceleration at Anthropic in October 2026” (published 5 November 2026, work conducted 12 August – 29 October 2026), cross-checked against Associated Press reporting via Fast Company (5 November 2026) and the aistockwire explainer of the report (6 November 2026). Every figure is METR’s or Anthropic’s own, with its uncertainty range kept beside it; nothing here is hands-on tested — this is a desk-researched explainer, and the article says so where it matters. Verification date: 7 November 2026.

Related reading on OpenAIMaster.ai: Anthropic’s S-1 Explained: Revenue, Loss and Control · Anthropic’s Agent Skills Just Went Open Source · The Best AI in the World Scores Just 12% on Real Work · Google’s New Engineering Exam for AI Agents

Leave a Reply

Your email address will not be published. Required fields are marked *