# DeepSeek Open-Sources DeepInfer as an Offline-Inference Release for Nvidia SM100 and Huawei Ascend 950

Times of AI Desk · 2026-10-20 · Infrastructure

[https://timesof.ai/2026/10/deepseek-open-sources-deepinfer-offline-inference-engine-nvidia-ascend](https://timesof.ai/2026/10/deepseek-open-sources-deepinfer-offline-inference-engine-nvidia-ascend)

> DeepSeek released DeepInfer under Apache 2.0 on Oct. 20, describing it as the inference framework behind its large-scale production services. The repo currently supports only DeepSeek-V4.1-Flash on Nvidia SM100 GPUs and Huawei Ascend 950 NPUs, and omits KV-cache, prefill/decode disaggregation and replica load balancing needed for a full online service. We omit DeepSeek's speed figures.

Serving software usually stays secret; DeepSeek has published an offline-inference release of the framework it says runs its production services, with Huawei Ascend as a supported target. **DeepSeek** announced **DeepInfer** on **Tuesday, October 20** and released it on GitHub under **Apache 2.0**. The README describes it as the high-performance inference framework behind DeepSeek's large-scale production services. For now it supports only **DeepSeek-V4.1-Flash**, loading the **V4.1-Flash-1017** checkpoint by default.

## What is in the release

DeepInfer runs on **Nvidia SM100** GPUs and **Huawei Ascend 950** NPUs. Documented features include DeepSeek's optimised kernels (FlashMLA, DeepGEMM, DeepEP and others), attention data-parallel and MoE expert-parallel scheduling, **DSpark** speculative decoding, an asynchronous scheduler, batch-invariant inference, deterministic sampling and structured outputs. DeepSeek says the project was originally built mainly on **vLLM**, with inspiration from SGLang and TensorRT-LLM.

It is an **offline-inference** release. The README lists what a complete inference service still needs and this release does **not** include: a KV-cache system, VE/prefill/decode disaggregation, and load balancing across data-parallel replicas.

We leave DeepSeek's published throughput and latency figures out of this piece; The Next Web, citing Bloomberg, notes they have not been independently tested. We covered the [1017 open-weight checkpoint](https://timesof.ai/2026/10/deepseek-v4-1-flash-1017-open-weights-mit) separately. DeepInfer arrives a day after MeriTalk reported the [DHS FASCSA order](https://timesof.ai/2026/10/dhs-fascsa-order-removes-deepseek-civilian-federal-it) removing DeepSeek from civilian federal IT (Federal Register notice filed Monday for Tuesday publication), and after reporting that [Huawei engineers are co-designing Ascend chips with DeepSeek](https://timesof.ai/2026/10/deepseek-huawei-ascend-chip-codesign-the-information).

## Limits

- Performance numbers in the README and secondary coverage are DeepSeek's; we do not treat them as verified here.
- Offline only: no KV cache, prefill/decode disaggregation or DP load balancing in this release.
- TNW says V4.1-Flash launched in September; the 1017 checkpoint we covered shipped October 17.

## Sources

- [GitHub: deepseek-ai/DeepInfer README](https://github.com/deepseek-ai/DeepInfer)
- [The Next Web: DeepSeek open-sources DeepInfer (October 20, 2026)](https://thenextweb.com/news/deepseek-deepinfer-open-source-inference-engine)
- [X (@deepseek_ai): DeepInfer announcement (October 20, 2026)](https://x.com/deepseek_ai/status/2112476496283775135)
- [Times of AI: DeepSeek V4.1-Flash-1017 open weights (October 20, 2026)](https://timesof.ai/2026/10/deepseek-v4-1-flash-1017-open-weights-mit)
- [Times of AI: DHS FASCSA order removes DeepSeek (October 20, 2026)](https://timesof.ai/2026/10/dhs-fascsa-order-removes-deepseek-civilian-federal-it)
- [Times of AI: DeepSeek–Huawei Ascend co-design (October 20, 2026)](https://timesof.ai/2026/10/deepseek-huawei-ascend-chip-codesign-the-information)