Artificial Analysis has launched a new leaderboard that tests leading AI models on a skill most benchmarks ignore: reading scientific images. The independent evaluator’s VIALS-AA, released on 6 November, ranks 19 frontier vision-language models on 161 tasks drawn from life-sciences research, from gel blots to microscopy slides. Google’s Gemini 4 Carbon leads at 64%, narrowly ahead of OpenAI’s GPT-6 Astra at 63% and Anthropic’s Claude Opus 5.5 at 61%. The headline, though, is cost: completing the same set of tasks can cost anywhere from two cents to $1.77.
What VIALS-AA measures
VIALS-AA is Artificial Analysis’s own implementation of Handshake AI’s VIALS benchmark, a dataset of visual questions about the images biotech researchers handle every day. The 161 tasks span eight domains, including gel blots, flow-cytometry plots, plasmid maps, phylogenetic trees, protein structures and small molecules.
Each question was written and reviewed by PhD-level scientists, and models must answer from the images alone. Scores use a pass-cubed rule: a model only earns a task if it answers correctly on all three attempts. Unlike a single-shot quiz, models work through the Stirrup harness, running code and inspecting images as they go, much as a researcher would.
A tight top three, a wide price gap
The top of the table is a near-tie. Gemini 4 Carbon, at its high setting, answered 64% of tasks correctly across all three repeats, with GPT-6 Astra at its maximum setting on 63% and Claude Opus 5.5, with fallback, on 61%. Carbon also leads in three of the eight domains.
With only three points separating the leaders, accuracy alone barely distinguishes the frontier — and that is where price takes over. GPT-6 Luna, at its maximum setting, completes the full task set for $0.02, while Grok 4.7, at its highest setting, costs $1.77 for the same work: a gap of nearly 99 times between models in the same performance tier.
The sharpest comparison is between GPT-6.1 Sol and Claude Opus 5.5. Sol scores 59% at about $0.06 per task; Opus 5.5 scores 61% at about $1.64. That is roughly 29 times the cost for a two-point edge in accuracy — a premium that is hard to justify unless those two points decide a result that matters.
How the models actually work the images
Artificial Analysis also tracked how models use their tools across 9,177 runs, and the styles differ sharply. Most tool calls were about visual evidence: 38.3% viewing images and 47.3% preparing or measuring them, by cropping, zooming or extracting measurements from pixels.
GPT-6 Astra and GPT-6.1 Sol spent only 5–6% of calls measuring images, against 25–27% for Claude Opus 5.5 and Claude Sonnet 5.5 — different routes to similar scores. At the other extreme, Gemini 3.8 Flash filled 86.7% of the cohort’s commentary calls, narrating instead of computing, across 67.5 calls per task.
The open-weights gap and the refusal problem
Open-weights models trail well behind. Kimi K3.1 Pro, the best of them, scored 45% — about 19 points below the leaders. The gap is a reminder that the strongest vision-language work still sits behind closed APIs, at least on scientific imagery, and that open models have the most ground to make up in specialist domains.
Fourteen tasks also produced safety refusals, and not because the questions were dangerous. Models misread scientific imagery as potentially sensitive content and declined to answer. It is a small share of the set, but it shows that vision models can still stumble on the very images researchers need them to read.
Our Read
The honest reading is that VIALS-AA crowns no clear winner. Three points separate the top three, and the scoring bar is strict enough that a single failed repeat costs a task. What the leaderboard really measures is value: cheaper models deliver nearly the same accuracy for a fraction of the price.
There is one comparison readers should not make. Fireworks runs the same VIALS dataset as a single-call test, where GPT-6 Astra leads at 49.9% — a different method from Artificial Analysis’s agentic, three-repeat scoring, so the numbers are not comparable. And with only 161 tasks in one specialist field, a single new domain could still reshuffle the table.
Why It Matters
Benchmarks shape which models labs and companies actually buy, and most of them test text. A benchmark that grades models on the images scientists use every day — and prices the answer — gives buyers a way to compare on value rather than on a single headline score.
If VIALS-AA catches on, expect vendors to start competing on cost per scientific task, not just on accuracy. For open-weights builders, the 19-point gap is the challenge to close; for buyers, the message is simpler — check the price tag before the ranking.
Background
Handshake AI published the original VIALS dataset in August 2026 and revised it in September, aiming to build evaluation infrastructure for real scientific image work. Artificial Analysis runs VIALS-AA continuously, and its earlier Intelligence Index v5, launched in late October, also put Gemini 4 Carbon narrowly ahead of its two rivals.
Source: Artificial Analysis launch article · Crypto Briefing report
0 Comments