DeepSeek announced DeepInfer in a four-post thread on X at 17:30 Beijing time on October 20, 2026. According to the thread, the framework powers DeepSeek-V4.1-Flash at scale with high throughput and low latency, and DeepSeek is sharing the production stack to help the community lower compute costs, reduce energy use and make AI more affordable. DeepSeek says inference code for future DeepSeek models and ongoing optimizations will also land in the same repository.

What is in the release

According to the DeepInfer README on GitHub, the framework currently supports only DeepSeek-V4.1-Flash, with DeepSeek-V4.1-Flash-1017 as the default checkpoint, and runs on NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs. The code is released under the Apache License 2.0.

The README lists optimized kernels, Attention DP and MoE expert parallelism, DSpark drafting with confidence-scheduled verification, SWA bounded replay, asynchronous scheduling, batch-invariant inference, deterministic sampling and structured outputs among its features.

The README says DeepInfer was originally built primarily on vLLM, with additional inspiration from SGLang and TensorRT-LLM.

DeepSeek also open-sourced deepseek-recipe, which the thread describes as its production token encoding and decoding component, aligned with the processing logic behind the official DeepSeek API. The deepseek-recipe README says it is a set of Rust libraries with Python bindings that renders DeepSeek V4 and V4.1 conversations into prompts and converts outputs into API response formats. That repository is MIT-licensed.

What is not included

The DeepInfer README states that this release provides offline inference and lists three components required for a complete inference service that are not included: a KV cache system, VE/Prefill/Decode disaggregation, and load balancing across data-parallel replicas for prefill and decode. The README adds that most of the engine functionality needed for disaggregation is already implemented. Teams looking to serve V4.1 Flash in production would need to supply those pieces themselves.

Performance

The README shows DeepSeek's own prefill and decode throughput charts rather than a table of figures. DeepSeek says prefill was measured on 4 SM100 GPUs and decode on 64, each within one scale-up domain, and provides a script to reproduce the sweep. These are DeepSeek's internal tests and have not been independently verified.