Replit’s 32-Agent Swarm Hit 18.9x — and Beat GPT-6.2

Human Verified

Share to: , and

A swarm of small robots assembling puzzle pieces together above a laptop

A swarm of 32 Claude agents sped up NumPy code by 18.9 times in two hours, against 3.7 times for a single agent, in Replit’s new test of AI teamwork. The surprise was not the speedup: OpenAI’s strongest solo model got worse as its team grew larger.

How the Test Worked

Replit built Optimization-Bench, a suite of performance tasks across 14 open-source libraries, and open-sourced it on GitHub. Each agent got eight virtual CPUs, 16 gigabytes of memory and a two-hour window, with network access strictly off and every run audited afterwards for compliance.

Six frontier models were tested alone and in swarms of 8, 16 and 32 agents, with no instructions on how to organise. The coordinator received the task but not a plan, so how each swarm divided the work and merged its branches was left entirely to the agents alone.

What the Swarms Did

Anthropic’s models improved as their teams grew, with the biggest jump on NumPy, where 13 separate workloads gave every agent something of its own. Opus swarms rose from 3.7 times solo to 18.9 times at 32 agents, and Sonnet swarms beat their solo runs every time.

GPT-6.2 Astra told the opposite story entirely. It was the strongest solo agent on every task, more than three times faster than Opus alone on NumPy, yet its swarms peaked early at eight agents and fell below their solo score at 32 on NumPy, SQLGlot and Git.

Our Read

The deciding factor was not talent but topology. The swarms that scaled kept coordination on a shared board that any agent could read later; the ones that stalled routed everything through a coordinator whose inbox grew heavier with every new agent added.

Opus workers sent 82 percent of their messages to the board, while GPT-6.2 Astra’s workers sent around 15 direct messages each to the coordinator, and fewer than 40 percent of its messages reached the board. Information that lived only in direct messages went stale before the team could use it.

The result echoes what Cursor and Cognition have reported: planners should stop implementing, workers should stop planning, and writes should stay single-threaded. Agents are already finding real flaws alone, so the open question now is which models can actually work together.

Why It Matters

Multi-agent systems are moving from demos to products — Replit’s own Missions feature launches swarms for users — so knowing which models cooperate is now a buying decision, not a curiosity. A model that is brilliant alone and chaotic in a team is a liability at scale.

The benchmark itself may matter more than the headline. Optimization-Bench rewards finding more speed, not finishing sooner, which makes it a genuine test of teamwork rather than a race. If other labs adopt it, swarm behaviour becomes comparable across models for the first time.

Background

Replit’s numbers are self-reported, from its own benchmark on its own platform, and the company sells the swarm product the test promotes. Claude has shown similar team behaviour before in published research, when 22 agents spent six weeks solving black-hole physics problems together.

Source: Replit engineering blog

Share to: , and

Faheem R. Khattak

Senior automation engineer and AI marketing journo covering LLMs, algorithms, analytics, core updates, Bing search, and technical SEO.

0 Comments

Leave a Reply

Optional Take Tag:

🛡️ Spam protection: Honeypot + Time-Trap Active

This site uses Akismet to reduce spam. Learn how your comment data is processed.