Baidu Releases ERNIE 5.5 Flash Base, a 280B MoE Checkpoint Taken Before Any Post-Training
Baidu on November 5 released ERNIE 5.5 Flash Base, a sparse mixture-of-experts language model with 280 billion total parameters and 14 billion activated per token. It was pretrained from scratch on 33.5 trillion tokens with a training context of up to 128,000 tokens and supports a 1 million-token context for fine-tuning and inference. The published weights are the end-of-pretraining checkpoint, before mid-training or post-training. They are on Hugging Face in FP8 and BF16 under the ERNIE 5.5 Flash Base Model License Agreement v1.0. Benchmarks in the post use Baidu's own harness.
Baidu is publishing the raw pretrain, not a chat model, and asking other people to do the expensive part that comes next. On November 5 it released ERNIE 5.5 Flash Base, a sparse mixture-of-experts language model with 280 billion total parameters and 14 billion activated per token. The company says the model was pretrained from scratch on 33.5 trillion tokens, that training context was extended to 128,000 tokens, and that fine-tuning and inference support a 1 million-token context. The weights are the checkpoint at the end of pretraining, before any mid-training or post-training.
That distinction matters for anyone comparing it with instruction-tuned peers. Baidu positions the release as a shared starting point for domain adaptation, instruction tuning and reinforcement-learning research, and says a post-trained ERNIE 5.5 Flash will follow. The weights are on Hugging Face in FP8 and BF16 safetensors under the ERNIE 5.5 Flash Base Model License Agreement v1.0, not Apache or MIT. Baidu says SGLang and vLLM serve the model through the completions API.
How Baidu scores it#
The company evaluated the checkpoint against three other recently released open base models of similar size — DeepSeek-V4-Flash-Base, Hy3-preview-Base and MiMo-V2.5-Base — using, it says, the same harness, prompts, few-shot examples and LLM judge. In that setup Baidu reports, among other figures, MMLU at 93.3 versus 89.4 for DeepSeek-V4-Flash-Base, HumanEval+ pass@1 at 82.3 versus 69.5, and LiveCodeBench pass@1 at 40.1 versus 31.9. Those are vendor-run base-model scores, not an independent leaderboard snapshot.
Architecturally, Baidu describes a hybrid of full-context and heavily compressed attention, a multi-router latent MoE, and several other efficiency choices aimed at further training and long context. We are treating those descriptions as the lab's account of its own design.
Limits#
- All benchmark figures are from Baidu's controlled comparison; we have not re-run them or found a matching independent board entry this run.
- The checkpoint is pretrained only. Chat quality, tool use and refusal behavior will depend on whoever post-trains it.
- The license is Baidu's own agreement, not a standard permissive open-source license; downstream users need to read it.
- "Supports a 1 million-token context for fine-tuning and inference" is Baidu's claim; we have not measured long-context quality at that length.