Google’s Gemini 4 Carbon has taken first place on Artificial Analysis’s rebuilt Intelligence Index, scoring 59 on version 5 of the ranking and overtaking Claude Opus 5.5, which led the new index at launch with 58. The result, published on 28 October 2026, came a day after Artificial Analysis released Index v5, its biggest upgrade since version 4 in January, in which 60 percent of the ranking now rests on private test sets or held-out answers. Carbon remains available only to selected users, with a 50 percent launch discount and no confirmed end date.
What Changed In Index v5
Index v5 adds two new benchmarks and upgrades three more, in a deliberate attempt to fight contamination, where models score well because they have seen test questions during training. FrontierCode 2.0, a private benchmark of real-world coding tasks graded on whether a repository maintainer would merge the agent’s change, and Terminal-Bench-Science, seventy agentic scientific research tasks run in a terminal, join the index for the first time.
Humanity’s Last Exam is replaced by HLE-Diamond, a refined 1,000-question version split evenly between reasoning and knowledge. The index now weights four areas equally at 25 percent each: knowledge work, coding, science and general capability.
AA-Omniscience, the knowledge and hallucination benchmark, was updated to version 1.1 with saturated and public questions removed, and CritPt, the research-level physics benchmark, is now scored on a private forty-question hard subset. Alongside the index, Artificial Analysis launched a Progress Index that places models released since November 2022 on one continuous capability scale.
Carbon’s Numbers, In Detail
Carbon leads four of the nine index evaluations: HLE-Diamond, Terminal-Bench-Science, AA-Omniscience and AutomationBench-AA. On the coding side, it is second, not first: with the Antigravity command-line tool, Carbon scores 59 on the Coding Agent Index, three points behind Claude Code with Claude Opus 5.5 at 62, and it ranks second on both Terminal-Bench 4.0, at 64 percent, and FrontierCode version 1.1, at 51 percent.
On knowledge work, Carbon scores 1,609 Elo on AA-Briefcase, a 126-point improvement over Gemini 4 Argon, but still behind Grok 4.7, Muse Wonder and Kimi K3.1 Pro. Its cost per Intelligence Index task, at standard pricing of $4 per million input tokens and $20 per million output tokens, is $7.76, about 23 percent less than Claude Opus 5.5 at $10.04, while using 123,000 output tokens per task against 173,000 for Opus.
One caution applies to every comparison: Carbon was measured at high reasoning effort, while Opus and GPT-6 Astra were measured at maximum effort. The hallucination picture is mixed in a revealing way. Carbon scores 54 on AA-Omniscience version 1.1, six points ahead of Opus, with accuracy rising ten points over Argon to 58 percent, but its hallucination rate rises from 4 percent to 11 percent.
That is still the second-lowest rate of any model measured, behind only Argon, compared with 50 percent for both Opus and GPT-6 Astra, a reminder that a higher overall score can hide a worse failure mode.
The Open-Weights Gap Widens
At the launch of Index v5, Claude Opus 5.5 led at 58, ahead of GPT-6 Astra at 57 and GPT-6.1 Sol at 55, with Gemini 4 Argon next at 50. The gap between the leader and the best open-weights model widened from three points under version 4.3 to twelve points under version 5, with Kimi K3.1 Pro, whose weights are not yet released, scoring 46.
Among models with released weights, DeepSeek V4.1 Flash and GLM-5.3 lead at 32, twenty-six points behind the leader. That widening gap is one of the most important findings of the new index, because it suggests the private-test-set design is exposing a real capability difference rather than a memorisation advantage.
Independent testing matters for the same reason in other fields, including safety evaluations such as DeepSeek’s safety tests, where outside researchers, not the model maker, set the questions. A ranking is only as trustworthy as its test set, and version 5 is explicitly designed to make that test set harder to game.
Not The Launch Story Again
This ranking is distinct from the launch of Carbon itself, which Google announced as a cybersecurity-focused frontier model with leading scores on vulnerability discovery and patching, as covered in Gemini 4 Carbon’s launch.
The launch story was about what Carbon is built to do; this story is about how it measures against every other frontier model on a rebuilt, contamination-resistant scale. The two stories share a model, not a conclusion.
Why It Matters
Frontier rankings shape purchasing decisions, research priorities and public perception, which is why the design of the ranking matters as much as its winner. A one-point lead for Carbon over Opus, measured at different reasoning efforts, is narrow and should not be read as a decisive verdict on which model is best for any particular job.
What matters more is that the new index, with most of its weight on private or held-out tests, still produces a clear leader, and that the open-weights gap grew sharply under the stricter design.
For developers, the practical takeaway is narrower: Carbon is cheaper per task than Opus at standard pricing, stronger on science and knowledge evaluations, and still second on coding agents. Choosing between them will depend on the task, the effort setting and the price, not on a single headline score, however prominently that score is reported.
Background
The Artificial Analysis Intelligence Index is a composite measure of model capability across tasks reflecting real-world usage. Version 5, released on 27 October 2026, is its biggest upgrade since version 4 launched in January 2026.
Source: Artificial Analysis Carbon ranking and Artificial Analysis Index v5
0 Comments