Tianyi Cui
@tianyi
Work on @DeepSeekHarness and other projects at @deepseek_ai Product Division.
DM is open. Opinions are my own.
加入 March 2007
491 正在關注 58.9K 粉絲
DeepSeek just open sourced its high-performance, large-scale inference framework DeepInfer in Apache-2.0 license.
Supported features:
- Optimized kernels, model and sampler runtime
- Parallelism: Attention DP / balanced CP, MoE EP + EPLB
- DSpark and confidence-scheduled verification
- SWA bounded replay
- Asynchronous scheduling (zero-overhead batch scheduler)
- Batch-invariant inference
- Deterministic sampling
- Structured output
顯示更多
