vLLM
@vllm_project
A high-throughput and memory-efficient inference and serving engine for LLMs. Join to discuss together with the community!
参加 March 2024
36 フォロー中 45.4K ファン
🚀 Qwen3.6-27B-NVFP4 is inference ready with vLLM on NVIDIA Blackwell GPUs.
This checkpoint is optimized for Blackwell and reduces GPU memory requirements by ~2.5x for local AI with open-source models.
🧠 27B params, Hybrid Attention
📊 NVFP4 evals: 86.3 on MMLU Pro, 85.5 on GPQA Diamond
🛠️ Exclusively supported on vLLM as the runtime engine
Get started from the Hugging Face checkpoint:
もっと見る
Fast, efficient local AI with open-source models just got easier.
Qwen3.6-27B-NVFP4 is now on @HuggingFace!
It's optimized for NVIDIA Blackwell GPUs & inference ready with @vllm_project.
The checkpoint reduces GPU memory requirements by approximately 2.5x for powerful 27B-parameter inference on your own hardware.
もっと見る
