Mia(@MiaAI_lab):Rejoice once again Nvidia 16 GB+ users! 🔥 Qwen3.8 27B should now runs even faster, thanks to an awesome PR from @niklaslenz_ai! exllamav3 has been upgraded to 1.6.0, and the MTP head changed to a DFlash2 draft model. In the RTX 5090, here're the speed improvements: Prose: 103 to 113 tok/s Code: 180 to 275 tok/s Structured: 206 to 366 tok/s You should notice speeds improvements on RTX 3090, 4090, and 5070 Ti, but I don't have official numbers, so pls test it and report!

Mia

@MiaAI_lab

Building with AI & LLMs // Running frontier open models locally

加入 July 2022

359 正在關注 41.8K 粉絲

Mia@MiaAI_lab

2026.10.10 14:18

Rejoice once again Nvidia 16 GB+ users! 🔥 Qwen3.8 27B should now runs even faster, thanks to an awesome PR from @niklaslenz_ai! exllamav3 has been upgraded to 1.6.0, and the MTP head changed to a DFlash2 draft model. In the RTX 5090, here're the speed improvements: Prose: 103 to 113 tok/s Code: 180 to 275 tok/s Structured: 206 to 366 tok/s You should notice speeds improvements on RTX 3090, 4090, and 5070 Ti, but I don't have official numbers, so pls test it and report!

156

轉發到社區

熱門用戶