DeepSeek Announces Full Open-Source Release of Internal Inference Framework DeepInfer

TechFlow News from DeepSeek, October 20: DeepSeek announced that it will fully open-source its internal inference framework, DeepInfer, and simultaneously release the production-grade token encoding and decoding component, deepseek-recipe, to help developers and enterprises optimize throughput, latency, and deployment costs for large-scale model inference. DeepInfer integrates high-performance operators and a deeply optimized inference engine, supporting capabilities such as distributed inference, speculative decoding, SWA Bounded Replay, and structured output. It is compatible with both NVIDIA and Huawei Ascend platforms. DeepSeek stated that it has already implemented high-throughput, low-latency large-scale inference services for DeepSeek V4.1 Flash based on DeepInfer. Future inference code for new models and subsequent optimizations will continue to be updated within this framework. Additionally, deepseek-recipe will maintain consistent processing logic with DeepSeek's official API, enabling the community to experience an API usage experience close to official deployment.