AJ(@ItsmeAjayKV):llama.cpp PR #27742 updated q8_0 KV was broken on Qwen3.8-Flash-Next when i tried for first time. Now it's fixed, so take a pull and rebuild. I was running f16 KV. Dropping to q8_0 should let me keep more experts on GPU (less CPU offload) and get more speed.

2026.08.27 05:52

llama.cpp PR #27742# updated q8_0 KV was broken on Qwen3.8-Flash-Next when i tried for first time. Now it's fixed, so take a pull and rebuild. I was running f16 KV. Dropping to q8_0 should let me keep more experts on GPU (less CPU offload) and get more speed.

AJ@ItsmeAjayKV

2026.08.26 19:01

Here is the first speed test of @Alibaba_Qwen Qwen3.8-Flash-Next, running on my 3090 😍 @UnslothAI UD-IQ3_XXS: 82GB Total memory: 88GB 24 GB VRAM + 64 GB RAM llama.cpp (pr #27742#) - Total usable context at f16 kv: 158K - Total VRAM usage 23GB Tested at depth up to 128k context. - Avg decode speed: 21 t/s - Peak decode speed: 33 t/s - Peak prefill speed: 168 t/s @ 32k All results were run without MTP or DFLASH

커뮤니티로 전달