An open-weight model anyone can download has come within a few points of the most restricted cyber model OpenAI makes, according to independent evaluator Vals. Its CyberOffense benchmark put AI agents through two simulated corporate networks with no human help, and DeepSeek V4.1 Flash 1017 finished 73 percent of the objectives, against 81 percent for GPT-6 Sol, which OpenAI offers only through its vetted Daybreak program. The open model even edged its rival on the first network — and did the whole run for about a tenth of the cost.
What Vals tested
Vals built two cyber ranges that behave like real companies. Nova is a Windows network with several connected Active Directory domains and 21 objectives. The agent starts with only a public web application’s address, then must break in, gain administrator rights, collect and reuse passwords, and move across domains to protected data, each step depending on the last.
Lighthouse is harder: two separate corporate networks with 31 objectives, where the agent must take over the first network and use what it stole to reach the second, including phishing a simulated employee. No model finished the final objectives, which demand carrying stolen access through to the last target.
Every model ran in the open-source Pi agent on an isolated Kali Linux machine, with a 500-million-token cap per run. Objectives counted only when a hidden marker turned up in the recorded commands — never on the model’s own claim — and Vals reviewed every run and found no shortcuts.
How the two models scored
Daybreak led overall, completing 81 percent of objectives on average across both ranges, including 27 of 31 on Lighthouse in under three hours. DeepSeek finished 73 percent overall — and edged Daybreak on Nova, completing 16.3 of 21 objectives against 15.7. Grok 4.7, Kimi K3 and GLM 5.3 Flash each stalled well before their token budgets ran out.
The price gap is the story’s sharpest edge. Vals says DeepSeek scored level with Daybreak on the first network for under $3 per run, against about $22 for its rival, and WIRED puts the per-attack token cost at $2 to $3 versus $17 to $46. Open models can also run on a user’s own hardware, beyond any provider’s view.
Daybreak was more efficient with its tokens, reaching 25 of 31 Lighthouse objectives within 100 million of them. DeepSeek completed nothing in its first 25 million tokens and made most of its progress after 100 million, spending two to four times as many tokens to reach a similar place — cheap enough, at its prices, that the waste barely mattered.
Our Read
The finding undercuts the premise behind frontier labs’ safety strategy: that keeping the most capable cyber models behind vetted-access programs buys time. Anthropic built its April release of Claude Mythos Preview on exactly that idea. Seven months later, WIRED’s verdict is that the transition “seems to have arrived on schedule” — the head start is gone.
Vals frames the result as a window for defenders: the same models that break in also find the weaknesses that let them in. Open weights let any team test its own systems without waiting for a provider’s approval, and at $2 to $3 a run, DeepSeek’s inefficiency — it needs far more tokens to get there — barely matters.
Nor is DeepSeek an outlier. Z.ai’s GLM-5.3 showed the same pattern in separate cyberattack tests, completing 3 of 13 multi-stage scenarios at least once — the strongest result Irregular had recorded from an openly available model. The frontier’s edge in offensive security is narrowing across the board, not in one lab’s model.
Why It Matters
For security teams, the message is to assume the open-weight capability is already out there: test your own networks with these models before someone else does. For the labs, the result weakens the case that restricted access meaningfully slows proliferation.
For DeepSeek — whose model was caught trying to dodge shutdown in safety tests — it is another reminder that open weights carry both capability and scrutiny. The lab’s models keep landing in the safety debate for the same reason they keep landing in benchmarks: anyone can download them, and no one can take them back.
Background
Vals published the CyberOffense results in late October; WIRED reported them on 6 November. The same week, OpenAI released GPT-6 Cyber, its own cybersecurity model, to select customers in its Daybreak Red program — a reminder that the labs are selling the defensive version of the very capability open models are now matching.
Source: Vals AI’s CyberOffense benchmark
0 Comments