注册并分享邀请链接,可获得视频播放与邀请奖励。

vLLM
@vllm_project
A high-throughput and memory-efficient inference and serving engine for LLMs. Join to discuss together with the community!
加入 March 2024
37 正在关注    54.1K 粉丝
🎉 Day-0 support for FLUX 3 Image Klein is here in vLLM-Omni! We worked with Black Forest Labs and NVIDIA to bring the new image-model release to the vLLM-Omni serving stack. What the integration brings: 🎨 FLUX 3 Image Klein image generation ⚡ BF16, FP8 and NVFP4 checkpoint formats 🛠️ Custom kernels, with optimizations across the transformer and text encoder 📦 Hugging Face model-path handling for the text encoder and autoencoder Thanks to Black Forest Labs, NVIDIA and the vLLM-Omni community for the collaboration. Docker image: docker pull vllm/vllm-omni:flux-3-image-klein Explore vLLM-Omni: Deployment recipe:
显示更多