Alexey Fateev(@superalesha):Qwen3.8 Flash Next just served a 262k token prompt with an image on my 4x3090. Decode at that depth: 67 tok/s. At 260K its 66.3, so the curve is basically FLAT. Almost every number you see for this model is 20-21 tok/s on a single 24GB card with experts offloaded to RAM on llamacpp. I went the other way: vLLM, all experts on GPU, FP8 KV. Full recipe below in my github, one command, raw runs included. 🧵A deep-dive technical thread. Let's go!

Alexey Fateev

@superalesha

⚡I benchmark local LLMs on 4x RTX 3090s. exact configs, tok/s, VRAM, and what broke. ❤️ - 2xDGX Spark 🚀96GB VRAM | Local AI

가입 January 2026

325 팔로잉 중 3.7K 팬

Alexey Fateev@superalesha

2026.08.28 11:05

Qwen3.8 Flash Next just served a 262k token prompt with an image on my 4x3090. Decode at that depth: 67 tok/s. At 260K its 66.3, so the curve is basically FLAT. Almost every number you see for this model is 20-21 tok/s on a single 24GB card with experts offloaded to RAM on llamacpp. I went the other way: vLLM, all experts on GPU, FP8 KV. Full recipe below in my github, one command, raw runs included. 🧵A deep-dive technical thread. Let's go!