註冊並分享邀請連結,可獲得影片播放與邀請獎勵。

vLLM
@vllm_project
A high-throughput and memory-efficient inference and serving engine for LLMs. Join to discuss together with the community!
加入 March 2024
37 正在關注    54.1K 粉絲
🎉 Day-0 support for FLUX 3 Image Klein is here in vLLM-Omni! We worked with Black Forest Labs and NVIDIA to bring the new image-model release to the vLLM-Omni serving stack. What the integration brings: 🎨 FLUX 3 Image Klein image generation ⚡ BF16, FP8 and NVFP4 checkpoint formats 🛠️ Custom kernels, with optimizations across the transformer and text encoder 📦 Hugging Face model-path handling for the text encoder and autoencoder Thanks to Black Forest Labs, NVIDIA and the vLLM-Omni community for the collaboration. Docker image: docker pull vllm/vllm-omni:flux-3-image-klein Explore vLLM-Omni: Deployment recipe:
顯示更多