Baidu has released the weights for ERNIE-5.5-Flash-Base, a 280-billion-parameter mixture-of-experts model that activates just 14 billion parameters per token. The release, published on Thursday, is Baidu’s first open-weights model in about 16 months, since the ERNIE 4.5 series in June 2025, and it arrives as a raw base model: a checkpoint taken at the end of pre-training, with no instruction tuning or safety alignment, ready for developers to fine-tune themselves. A post-trained version will follow, Baidu says.
What Baidu Actually Released
The model was pre-trained from scratch on 33.5 trillion tokens, with a training context that grew to 128,000 tokens and support for a one-million-token context in fine-tuning and inference. It has 43 Transformer layers, and all but the first use a mixture-of-experts block with 512 routed experts, ten of which activate per token, plus one shared expert.
The weights are available on Hugging Face in FP8 and BF16 formats, and the public repository went live on Thursday. The model expects a beginning-of-sequence token at the start of every input, and Baidu documents support for the SGLang and vLLM serving frameworks, so the checkpoint can be run through a standard completions API.
The Efficiency Bet
The design is built around one idea: a base model meant for further training should be cheap to train and cheap to run. Most of its 43 layers use a heavily compressed form of attention, squeezing every 128 tokens into a single cached entry, while six layers keep full token-level access so long-range performance holds up.
The result is a much smaller key-value cache, the memory that usually limits long-context work. Baidu also replaced the usual multi-token prediction module with a design that reuses the model’s own final layer to predict future tokens, and it added a learnable output head that suppresses implausible tokens while keeping several plausible ones in play.
The Scores Baidu Is Claiming
Baidu compared the model against three rival base models of similar size — DeepSeek-V4-Flash-Base, Tencent’s Hy3-preview-Base and Xiaomi’s MiMo-V2.5-Base — running all four through the same evaluation harness, prompts and few-shot examples. On Baidu’s count, ERNIE 5.5 Flash Base came first in 24 of 30 benchmarks across seven categories, with its strongest results in exams and code.
The comparison is flattering by design: Baidu ran the evaluation itself, using the same harness for all four models, and the scores have not been independently reproduced. That caveat matters, because every number in the table comes from the company selling the model.
Still, the gaps are wide in places: 93.3 on MMLU against 89.4 for DeepSeek’s model, 82.3 on HumanEval+ against 69.5, and 91.7 on the RepoQA long-context test against 81.0. ERNIE did not win everywhere — DeepSeek’s model took ChineseSimpleQA and two long-context tests, and Tencent’s model won one coding and one reasoning test.
Original Analysis: What This Changes
The release lands in the middle of an open-weights race among Chinese labs. Tencent’s Hy4 arrived on Tuesday with vision and computer use, and Moonshot put Kimi K3.1 Pro’s weights online this week. Baidu is now the third major Chinese lab in as many days to hand developers a frontier-scale checkpoint.
The pattern is clear: the open model is becoming the distribution strategy, not the afterthought. But Baidu’s gift comes with the tightest strings in this race. Where the ERNIE 4.5 series shipped under the permissive Apache 2.0 licence, this model uses a custom licence instead.
Reports on the release say that licence bars use in the European Union, the United Kingdom and South Korea, and requires written permission for commercial use above $50 million in annual revenue. For a model pitched as a starting point for the community, the exclusions are large enough to shape who can actually build on it.
Why It Matters
A 280-billion-parameter base model with a million-token context, released openly, lowers the cost of entry for anyone building models on top of it. Research teams and smaller companies can now start from a checkpoint that cost Baidu 33.5 trillion tokens of training, instead of paying for pre-training themselves, and the model’s efficiency-first design makes that starting point cheaper to run.
The licence is the counterweight. A model that cannot be used in Europe, Britain or South Korea, and needs Baidu’s written permission for larger commercial users, is open in a narrower sense than the Apache-licensed releases it follows. Developers weighing this checkpoint against DeepSeek’s or Tencent’s open models will have to read the licence before they read the benchmarks.
Background
Baidu last open-sourced a model of this scale in June 2025, when the ERNIE 4.5 series arrived under Apache 2.0. The ERNIE 5.0 and 5.1 generations that followed stayed closed, and Baidu’s return to open weights comes as Chinese labs race to put their largest models in developers’ hands — Moonshot’s Kimi weights went public earlier in the week.
Source: Baidu’s ERNIE 5.5 Flash release and Hugging Face model page
0 Comments