Niklas(@niklaslenz_ai):Thanks for merging, @MiaAI_lab! A few details for anyone trying it: • It's opt-in. One line in .env: DRAFT=dflash2. With DRAFT unset, nothing changes. • One request, end to end: 155 → 199 tok/s. • The lead shrinks with context. From ~64k it is level with MTP on the same engine (exllamav3 1.6.0), slightly behind at 248k. • Quality held: MMLU 80.0 → 79.5%, GSM8K 96 → 96%. • One RTX 5090, Linux, 4.0 bpw, thinking off, median of 5 runs. The Windows path is built but not run yet. Will try to get something working on my 5060 16GB as well.

2026.10.10 14:27

Thanks for merging, @MiaAI_lab! A few details for anyone trying it: • It's opt-in. One line in .env: DRAFT=dflash2. With DRAFT unset, nothing changes. • One request, end to end: 155 → 199 tok/s. • The lead shrinks with context. From ~64k it is level with MTP on the same engine (exllamav3 1.6.0), slightly behind at 248k. • Quality held: MMLU 80.0 → 79.5%, GSM8K 96 → 96%. • One RTX 5090, Linux, 4.0 bpw, thinking off, median of 5 runs. The Windows path is built but not run yet. Will try to get something working on my 5060 16GB as well.

Mia@MiaAI_lab

2026.10.10 14:18

Rejoice once again Nvidia 16 GB+ users! 🔥 Qwen3.8 27B should now runs even faster, thanks to an awesome PR from @niklaslenz_ai! exllamav3 has been upgraded to 1.6.0, and the MTP head changed to a DFlash2 draft model. In the RTX 5090, here're the speed improvements: Prose: 103 to 113 tok/s Code: 180 to 275 tok/s Structured: 206 to 366 tok/s You should notice speeds improvements on RTX 3090, 4090, and 5070 Ti, but I don't have official numbers, so pls test it and report!