Eight Tech Giants Unite Behind Open Inference Engine SGLang

Human Verified 0

Share to: , and

A data centre hall with server racks and glowing network lines

LMSYS Org has launched the SGLang Consortium, bringing eight of the biggest names in computing together to steer the open-source inference engine that runs some of the world’s largest AI deployments. Founding members NVIDIA, AMD, Intel, Google, Meta, Cloudflare, Modal and RadixArk will work with SGLang’s maintainers to shape the project’s roadmap, the nonprofit announced on November 12 at the first SGLang Summit in San Francisco. The move puts rival chipmakers and cloud giants on the same committee at a moment when inference — running finished models for real users — is set to swallow roughly two-thirds of all AI computing.

What the consortium will do

The consortium is a standing forum, not a new product. Each founding member will name a technical representative and join two roadmap meetings a year, contributing engineering expertise, deployment feedback, testing resources or joint development. Working groups can collaborate between meetings, and LMSYS says it will publish updates on the roadmap’s direction and progress.

The work is organised around four areas: support for frontier models, reasoning and multimodal workloads, distributed inference, and hardware optimisation across the growing list of chips SGLang supports. SGLang’s maintainers keep final say over technical decisions and which code is accepted, and the project stays open to the whole community.

Engineers discussing an abstract network diagram on a wall screen

Why inference, why now

Inference is where the money is moving. Deloitte expects it to account for roughly two-thirds of all AI compute in 2026, up from about half in 2025, as the industry shifts from training models to running them for millions of users. The engine that serves a model decides how fast and how cheaply that compute runs on each company’s hardware.

SGLang already sits at the centre of that shift. The full-stack engine, part of the PyTorch ecosystem, runs on hundreds of thousands of GPUs and powers some of the largest production inference deployments in the world, at companies including Meta, Microsoft, Google, Amazon, Thinking Machines Lab, Cloudflare, Reflection and Modal.

The hardware angle is what makes the member list matter. Google’s eighth-generation TPUs are the newest accelerators in the consortium’s orbit, and the group’s hardware optimisation work spans the chips its members build — a spread that matters as buyers look beyond a single supplier.

Our Read

The striking thing is who is sitting together. NVIDIA, AMD and Intel compete on accelerators, and Google and Meta compete on models and clouds, yet all five have signed up to steer the same open engine. That is a bet that inference software is becoming shared plumbing, not a competitive edge — and that no vendor can optimise it alone.

It is also a promise rather than a product. The announcement carries no funding figures, no performance targets and no dates for new features, and the maintainers keep control of the code. The test will be whether twice-yearly meetings and working groups produce faster serving and lower costs, or a logo slide. The line-up is serious; the mechanism is untested.

The direction of travel is clear, though. Open serving tooling keeps gaining ground: Google open-sourced its XLA serving engine, and SGLang’s own users already include most of the industry’s biggest spenders. A shared roadmap for the engine they all run on is the logical next step, not a surprise.

Why It Matters

If the consortium works, the payoff is cheaper AI for everyone downstream. Lower inference costs make AI more accessible to organisations everywhere, in NVIDIA’s own words, and they decide whether the wave of agents and assistants can run at a price users will pay. The economics are already reshaping valuations: fal doubled to $16 billion on the inference bottleneck alone.

For the chipmakers, the stakes are different. Whoever’s hardware SGLang runs best on wins a share of the two-thirds of AI compute that inference will soon represent. A consortium that writes the roadmap together at least keeps that contest on engineering merit rather than lock-in — which is exactly what the smaller members, and the open-source community, will be watching.

Background

LMSYS Org, the nonprofit behind SGLang, FastChat, Vicuna and Chatbot Arena, develops open and scalable models and systems. The consortium was announced in the opening keynote of the first SGLang Summit, held November 12–13 at San Francisco’s Fort Mason Center, with more than 100 speakers from across hardware, models and infrastructure.

Source: the consortium announcement and SGLang.

Share to: , and

Faheem R. Khattak

Senior automation engineer and AI marketing journo covering LLMs, algorithms, analytics, core updates, Bing search, and technical SEO.

0 Comments

Leave a Reply

Optional Take Tag:

🛡️ Spam protection: Honeypot + Time-Trap Active

This site uses Akismet to reduce spam. Learn how your comment data is processed.