GitHub has detailed Project Rubicon, a rebuilt Git storage architecture designed for a world where coding agents, not people, do most of the pushing. In internal testing, the new system supported 5,000 concurrent agents working on a single repository, handling 7,200 pushes a second while keeping read latency low. The architecture is in limited private preview now, with a broader rollout planned for early next year. GitHub says the rebuild is the foundation for agent-driven development at the scale of hundreds of millions of developers.
What Rubicon changes
The design splits Git storage into parts that scale on their own. Git objects live as packs in Azure Blob Storage, while references and the pack manifest sit in a separate replicated log service GitHub calls Ref-CAS. Virtual replicas serve reads and writes from caches that can be rebuilt at any time. Developers still push, fetch, and branch as before.
The insight is that very little of a push needs ordering. Git objects are immutable and named by their contents, so two writers can store them at once without agreeing on anything. Only the reference update needs agreement, and that update is tiny. Ref-CAS commits each one through a quorum, so most pushes replicate in parallel instead of queuing.
Why Git needed rebuilding
Agents have changed what a busy repository looks like. GitHub says its busiest repositories now receive at most dozens of writes a second, in short bursts, but it expects those bursts to become a sustained rate within a year. One repository’s daily requests grew about fourfold between March and May, from 44 million to 181 million a day.
Every push also sets off more reads, as tests run, deployment systems pull the new commit, and other agents fetch it for context. The load is spreading fast: Copilot now runs inside Xcode’s built-in assistant and in JetBrains Air, putting agents in editors far beyond VS Code.
What comes next
The new architecture is available in a limited private preview now, with a broader rollout early next year. GitHub is also working on cheaper agent operations: reading files straight from the server without a clone, landing multi-file changes through one atomic commit API, and real-time event streams agents can subscribe to.
Rubicon is one half of GitHub’s agent rebuild. The other half is Copilot itself: GitHub is routing each coding task to the model that fits it best, rather than running everything through one model. Storage that scales and routing that adapts are the two halves of the same bet.
Our Read
This is GitHub admitting, in engineering terms, that agents are now its most important users. Humans push in bursts and go home; fleets of agents push around the clock, and every push fans out into tests, deploys, and more agent reads.
The design answers that directly: only the tiny reference update is ordered, and everything else scales sideways. It is the same problem Meta’s Conductor agents face, running coding agents in parallel, and GitHub is solving it at the storage layer rather than the agent layer.
The numbers come with a caveat: 5,000 agents and 7,200 pushes a second are internal test results, not production load, and the broader rollout is still months away. Fast storage means little if the code agents write cannot be trusted, which is why Mistral’s SkipLabs deal, buying verification for AI-written code, matters as much as the pipes underneath it.
Why It Matters
If agents write a growing share of the world’s code, the systems underneath them have to keep up. Rubicon removes the bottleneck that would have capped how many agents can work on one repository at once, without asking developers to change how they work. Every platform hosting agent-written code faces the same test, and GitHub set the bar.
Background
GitHub announced the architecture at its Universe conference last week, after months of building it underneath the live service. The work sits alongside a wider shift toward agents that write production code, from open-weight models built for agentic coding to the coding assistants now embedded in every major editor. Rubicon is the infrastructure catching up with that shift.
Source: GitHub’s engineering post
0 Comments