Mia
@MiaAI_lab
Building with AI & LLMs // Running frontier open models locally
参加 July 2022
359 フォロー中 41.8K ファン
Rejoice once again Nvidia 16 GB+ users! 🔥
Qwen3.8 27B should now runs even faster, thanks to an awesome PR from @niklaslenz_ai!
exllamav3 has been upgraded to 1.6.0, and the MTP head changed to a DFlash2 draft model.
In the RTX 5090, here're the speed improvements:
Prose: 103 to 113 tok/s
Code: 180 to 275 tok/s
Structured: 206 to 366 tok/s
You should notice speeds improvements on RTX 3090, 4090, and 5070 Ti, but I don't have official numbers, so pls test it and report!
もっと見る
