vLLM
@vllm_project
A high-throughput and memory-efficient inference and serving engine for LLMs. Join to discuss together with the community!
参加 March 2024
36 フォロー中 50.2K ファン
Excited to see @peano_ai run full-parameter RL for @XiaomiMiMo 310B MiMo-V2.6 on TPUs, across 1,000+ TPUs. 🚀🚀
vLLM drives the rollouts, bitwise-matched with the trainer in validation. All 310B params move across the ICI fabric in under 2s.
🔗
もっと見る
We enable full-parameter RL on TPUs: MiMo-V2.6 at 310B, plus other stable training runs of 1,000+ steps across 1,000+ TPUs.
With JAX, scaling up is a config change, not a rewrite.
We built on that with optimized vLLM inference for faster rollouts and full bitwise trainer–sampler agreement in validation.
Trainer and sampler share one TPU ICI fabric. All 310B MiMo-V2.6 parameters transfer in <2 seconds.
もっと見る
