DeepSeek 宣布将其内部推理框架 DeepInfer 完全开源，用于支撑 DeepSeek-V4.1-Flash 的高吞吐、低延迟推理。框架支持 NVIDIA 和华为昇腾硬件，提供分布式推理、投机解码、SWA Bounded Replay 和结构化输出能力，同时还发布了与其官方 API 完全对齐的生产级分词组件 deepseek-recipe。
推理框架兼顾 NVIDIA 与华为昇腾硬件，读者可以借此了解 DeepSeek 生产栈在成本和能效上的实际做法。
DeepSeek is open-sourcing DeepInfer, its internal framework for serving models at scale.
Inference frameworks coordinate compute clusters to shape AI throughput, latency, cost, and energy efficiency. DeepSeek says DeepInfer powers high-throughput, low-latency inference for V4.1 Flash.
Supporting both Nvidia and Huawei Ascend hardware, DeepInfer enables distributed inference, speculative decoding, SWA Bounded Replay, and structured outputs. Alongside the framework, DeepSeek also released deepseek-recipe, a production-grade tokenization component fully aligned with its official API.
🚀 Introducing DeepInfer, our in-house inference framework, now fully open source! Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable. 🔗 http://github.com/deepseek-ai/DeepInfer 1/4在 X 查看被引用的帖子
来源：X.PIN · x.com