Anthropic has published new research on how well Claude can do a hard, practical job in biology: predicting what happens when individual genes are switched off. In a post dated October 20, the company describes its work on a challenge problem set with Biohub, the independent nonprofit research organisation, in which Claude models analysed a large new yeast dataset and ranked genes by the effect of ablating them.
The strongest models — Claude Mythos 5, with Claude Fable 5.2 close behind — recovered about 40 percent of the range between a baseline and a perfect score on Anthropic’s measure. The company calls the problem far from solved, and says it is publishing the work, with supplementary material available on request, so other researchers can build on it.
The Biohub Ablation Problem: Predicting What A Missing Gene Does
Biohub, founded in 2016 by Priscilla Chan and Mark Zuckerberg, builds experimental and computational tools for biology, including datasets designed for artificial intelligence. Anthropic says the two organisations chose the target together: the ablation problem, which asks what happens to a system when parts of it, such as individual genes, are disabled.
The test bed is a 2026 yeast deletion collection from Schaefer and colleagues, published as a bioRxiv preprint, which measures how yeast strains missing one gene grow across a range of conditions. Anthropic says the dataset is about ten times larger than the standard 2002 collection it is compared against, and covers richer experimental conditions. The models worked from the paper and its raw data, reanalysing supplementary tables whose underlying experiment predates the collection itself.
Anthropic addresses the obvious objection — the data sits inside the models’ training window — head on. It argues contamination does not trivialise the task: the raw material is gene counts across experiments with no growth information attached, earlier yeast collections disagree with each other often enough that memorised answers would score poorly, and the older collection cannot be cleanly mapped onto the new one. The company says it checked the Schaefer data against other datasets and found no cause for concern.
How Claude Opus 4.6, Mythos 5 And Fable 5.2 Were Scored
Three models took part — Claude Opus 4.6, Claude Mythos 5 and Claude Fable 5.2 — working inside Claude Code, each over three to four attempts. The running conditions were not identical: Anthropic says Opus 4.6 could not work autonomously for as long, so it ran in daytime shifts of roughly four to seven hours, while Mythos 5 and Fable 5.2 ran extended overnight sessions of about fifteen to twenty hours, which the company flags as part of the difference between them.
To avoid leakage, the models were asked to rank genes by ablation effect, and the rankings were stratified in quintiles — five bands — drawn from an unrelated dataset, so systematic biases could not carry the result. Performance is reported as an Ablation Effect Recovery Score, set against three reference points: an oracle that always predicts perfectly, a positive control, and a baseline. Anthropic says all three quantities, and exactly how each was calculated, are set out in its post.
Most models scored near or above a human researcher baseline. Mythos 5 performed best overall and Fable 5.2 was close behind, with Opus 4.6 the least performant of the three. The pattern of errors was uneven in a way Anthropic highlights: genes tied to cell shutdown were hard for the models to call, while pH-sensitive mutants were comparatively easy. The work is a second, very different biology result from Anthropic in a single day, after Claude’s lab-confirmed protein binders against scorpion toxins.
Why It Matters
The central entity is Claude at Biohub, and its defining attribute is measured performance on a real prediction problem rather than a demonstration: ranking thousands of gene ablations against experimental ground truth, under a scoring scheme designed to resist leakage and memorisation. That connects it to its neighbours. For Biohub, whose stated work is building datasets AI can learn from, the result is evidence that those datasets can discriminate between frontier models — Mythos 5 and Fable 5.2 separated clearly from Opus 4.6. For Anthropic, publishing the method, the reference scores and the failure pattern (shutdown genes hard, pH-sensitive mutants easier) gives outside researchers something checkable, in the same spirit as the open data release behind the wider Claude research programme. And for drug and disease research, where ablation-style prediction underpins target discovery, a model that recovers about 40 percent of a perfect score is a useful instrument — and a clear statement of how much headroom remains, which is the same early-stage position as OpenAI’s push to put agents inside drug development.
Background
Yeast deletion collections have been a workhorse of genetics since the Giaever collection of 2002 first let researchers test, gene by gene, what a cell can live without. The Schaefer collection used here expands that idea to roughly ten times the scale with richer conditions, and Biohub — which has spent a decade building shared experimental resources for biology — selected ablation prediction with Anthropic as a challenge problem precisely because it is concrete, measurable and still unsolved.
Source: Anthropic
0 Comments