vLLM
@vllm_project
A high-throughput and memory-efficient inference and serving engine for LLMs. Join to discuss together with the community!
Joined March 2024
36 Following 50.3K Followers
đ Meet vLLM-Omni v0.22.0, a major upgrade for omnimodal world models and production-grade multimodal serving.
đ Day-0 @NVIDIAAI Cosmos 3 world models: text, image, audio, video, and action, in and out.
đ¤ Robot serving: DreamZero + OpenPI realtime API.
đī¸ Production TTS: Qwen3-TTS, Qwen3-Omni, VoxCPM2 and more.
đ¨ Faster image/video/diffusion: Wan 2.2, HunyuanVideo 1.5, LTX-2.3.
⥠Broader quantization (FP8/INT8, MXFP4/MXFP8, W4A16, ModelOpt) and hardware coverage.
339 commits, 124 contributors, 52 of them new. Thank you all. đ
đ
Show more
