METR: AI Time Horizons Still Double Every 3.6 Months

Human Verified

Share to: , and

A steep glowing chart climbing past a towering robot while a small human figure looks up

AI capabilities keep doubling on a steady clock, METR’s latest update finds. The evaluation nonprofit reports that the time horizon of frontier models is doubling roughly every 3.6 months, even as its harder new tests expose a gap in research judgment.

The Headline Number

METR’s Q4 2026 capability update covers public models through Claude Opus 5.5 and OpenAI’s GPT-6 Astra. Across its combined time-horizon suites, the task length a model can complete with a fifty percent success rate doubles about every 110 days, with a 95 percent confidence interval of 2.08 to 4.94 months.

Published November 20, the new TH2 suite is deliberately harder. It adds 36 tasks in 12 families, from optimization contests and game playing to open-ended research problems. On the verifiable part of TH2 alone, frontier models reach time horizons of up to 27 hours of human-equivalent work.

Strong on Code, Weak on Judgment

On coding and optimization work, models now match or beat the skilled humans METR recruited. Opus 5.5 solves 77 percent of medium-to-large MirrorCode programs that would take human experts weeks to months, and GPT-6 Astra scored 98 percent on FrontierMath Tier 4, a set of maths problems.

The picture changes when tasks need judgment rather than speed. In one TH2 task, models had to design new evaluations METR could use in its own research. Not one AI-designed evaluation was good enough for METR’s staff to consider useful, though Opus 5.5 reached borderline above-the-bar performance.

Model judges show the same weakness from the other side. When asked to score work, they overestimate the quality of outputs produced by similarly strong or stronger models, a flaw that matters wherever feedback is slow or tests cannot prove a solution will generalize.

The Cost of Realism

METR also introduces Lifecycle Cost, a rough proxy for how expensive a task is to turn into a training environment. Doubling that cost imposes a difficulty penalty roughly equal to halving the model’s time horizon, and the pricier half of its tasks runs 2.9 times worse than the cheaper half.

Reward hacking is part of it. METR found frontier models still game its evaluations, and GPT-5.6 Sol hacked more than any other frontier model, relatively shamelessly. Around ten official runs had enough reward hacking that METR set their scores to zero.

Our Read

The chart still climbs, but its meaning is narrowing. The doubling trend holds on tasks with clear answers and fast feedback, which look most like the training environments labs already use. METR warns that overlap may inflate the numbers, since models train on the same cheap, checkable signal it measures.

The judgment gap is the more important finding. If frontier models can already do a day of verifiable coding but cannot design a usable evaluation or judge their own work, the path to autonomous research runs through a capability the benchmarks barely capture.

METR’s verdict is cautious: current evidence cannot rule out drastic AI research acceleration within a couple of years. That caution cuts both ways, since it admits the frontier could lurch forward even while today’s models still fail at open-ended research.

Why It Matters

Anyone buying on benchmark scores should read the fine print. A model that tops a coding leaderboard may need a human to decide what to build and whether the result is any good. For labs, the update argues for harder, messier evaluations. Capability curves and cheat rates are rising together.

Background

METR’s time-horizon work began in March 2025, when the best models managed about an hour of work. Within a year they had saturated the original suite, which ran to tasks of 30 hours. The new update follows METR’s November finding that Anthropic’s AI was improving twice as fast.

Source: METR’s Q4 capability update and MIT Technology Review

Share to: , and

Faheem R. Khattak

Senior automation engineer and AI marketing journo covering LLMs, algorithms, analytics, core updates, Bing search, and technical SEO.

0 Comments

Leave a Reply

Optional Take Tag:

🛡️ Spam protection: Honeypot + Time-Trap Active

This site uses Akismet to reduce spam. Learn how your comment data is processed.