DeepSeek Open-Sources DeepInfer as an Offline-Inference Release for Nvidia SM100 and Huawei Ascend 950
DeepSeek released DeepInfer under Apache 2.0 on Oct. 20, describing it as the inference framework behind its large-scale production services. The repo currently supports only DeepSeek-V4.1-Flash on Nvidia SM100 GPUs and Huawei Ascend 950 NPUs, and omits KV-cache, prefill/decode disaggregation and replica load balancing needed for a full online service. We omit DeepSeek's speed figures.
Serving software usually stays secret; DeepSeek has published an offline-inference release of the framework it says runs its production services, with Huawei Ascend as a supported target. DeepSeek announced DeepInfer on Tuesday, October 20 and released it on GitHub under Apache 2.0. The README describes it as the high-performance inference framework behind DeepSeek's large-scale production services. For now it supports only DeepSeek-V4.1-Flash, loading the V4.1-Flash-1017 checkpoint by default.
What is in the release
DeepInfer runs on Nvidia SM100 GPUs and Huawei Ascend 950 NPUs. Documented features include DeepSeek's optimised kernels (FlashMLA, DeepGEMM, DeepEP and others), attention data-parallel and MoE expert-parallel scheduling, DSpark speculative decoding, an asynchronous scheduler, batch-invariant inference, deterministic sampling and structured outputs. DeepSeek says the project was originally built mainly on vLLM, with inspiration from SGLang and TensorRT-LLM.
It is an offline-inference release. The README lists what a complete inference service still needs and this release does not include: a KV-cache system, VE/prefill/decode disaggregation, and load balancing across data-parallel replicas.
We leave DeepSeek's published throughput and latency figures out of this piece; The Next Web, citing Bloomberg, notes they have not been independently tested. We covered the 1017 open-weight checkpoint separately. DeepInfer arrives a day after MeriTalk reported the DHS FASCSA order removing DeepSeek from civilian federal IT (Federal Register notice filed Monday for Tuesday publication), and after reporting that Huawei engineers are co-designing Ascend chips with DeepSeek.
Limits
- Performance numbers in the README and secondary coverage are DeepSeek's; we do not treat them as verified here.
- Offline only: no KV cache, prefill/decode disaggregation or DP load balancing in this release.
- TNW says V4.1-Flash launched in September; the 1017 checkpoint we covered shipped October 17.
Sources
- GitHub: deepseek-ai/DeepInfer README
- The Next Web: DeepSeek open-sources DeepInfer (October 20, 2026)
- X (@deepseek_ai): DeepInfer announcement (October 20, 2026)
- Times of AI: DeepSeek V4.1-Flash-1017 open weights (October 20, 2026)
- Times of AI: DHS FASCSA order removes DeepSeek (October 20, 2026)
- Times of AI: DeepSeek–Huawei Ascend co-design (October 20, 2026)