Tianyi Cui(@tianyi):DeepSeek just open sourced its high-performance, large-scale inference framework DeepInfer in Apache-2.0 license. https://t.co/W2cfi6O9LO Supported features: - Optimized kernels, model and sampler runtime - Parallelism: Attention DP / balanced CP, MoE EP + EPLB - DSpark and confidence-scheduled verification - SWA bounded replay - Asynchronous scheduling (zero-overhead batch scheduler) - Batch-invariant inference - Deterministic sampling

Tianyi Cui

@tianyi

Work on @DeepSeekHarness and other projects at @deepseek_ai Product Division. DM is open. Opinions are my own.

Joined March 2007

491 Following 58.9K Followers

Tianyi Cui@tianyi

2026.10.20 09:08

DeepSeek just open sourced its high-performance, large-scale inference framework DeepInfer in Apache-2.0 license. Supported features: - Optimized kernels, model and sampler runtime - Parallelism: Attention DP / balanced CP, MoE EP + EPLB - DSpark and confidence-scheduled verification - SWA bounded replay - Asynchronous scheduling (zero-overhead batch scheduler) - Batch-invariant inference - Deterministic sampling - Structured output