YES, FUCKING YES! I DID IT !!! In my last post, I heavily trashed Qwen3.8 Flash Next, claiming it couldn't do anything at all. I managed to get awesome speed out of it, but in nearly 3 hours it couldn't even generate a simple Pagoda Garden. I had no clue what was going on. Turns out the issue was with my custom quant, which I built myself to fit 256k context + vision into 4x3090 with good throughput. @_cpatonn released a properly calibrated AWQ INT4 of this model, but it was larger than mine and didn't fit into my 96GB as-is. CUDA OOM - sounds familiar to everyone, right? I decided to tackle it and make something that actually works. A full day of work, interrupted only by 5 hours of sleep. To fit 262K context, I had to un-fuse the fused layers and quantize all non-sensitive projections to FP8. vLLM allows only 1 quantization scheme per fused layer, which turned into writing a custom loader and 13 launch attempts 😭 Attempt 12 finally loaded, and I was so hyped. But the model's output was pure gibberish, total mush. Turns out TP4 was slicing the fused QKV matrix in the wrong places, for fuck's sake. Attempt 13 fixed the slicing. 262K fit. The model kept spitting garbage on any complex task. Short answers were fine, a 40K needle test was fine, but ask for a three.js scene - and you get 16,384 tokens of complete shit. I was devastated. The root cause of all this was just mind-bogglingly fucked up. 48 tiny tensors, shared_expert_gate.weight, ~5 KB each. My build script quantized them to FP8, but the runtime read them as BF16 without applying the scale 🤡 The gate expects values around 0.077, but was receiving 448 🤡🤡 Eat up, my dear friend. And this gate determines how much the shared expert contributes to every single token. I restored all 48 from the original checkpoint - and the pagoda scene, which I had spent hours grinding on and ran dozens of times - just passed twice in a row. Wanna know how mindblown I was? Exactly. Then the model swallowed 258,008 real prompt tokens with an image and nailed the needle test. Decode at 63-68 tok/s at 220 W per GPU - identical to what the broken build pushed. Video is the proof. One prompt, two sets of weights: reference BF16 vs. my Franken-quant running on 4 gaming GPUs from 2020. One-shot, zero fixes. Mine finished first, and the scene came out tighter. It's just 1 prompt, so I won't call it a benchmark. 24 hours of my life for 240 KB of broken tensors 🤡🤡🤡

2 hours 43 minutes. 442 steps. 5,406 lines of code. The page still does not load 💀 I gave Qwen3.8 Flash Next my standard bench prompt: a voxel Japanese pagoda garden in Three.js, 16,500 voxels, modular source, run it and screenshot it when it's done. Other models finish this one easy. It never opened a browser. Never started the dev server. Never counted a voxel. It spent the entire run fighting its own export statements. petals.js got rewritten 19 times. Not patched, rewritten from scratch every time. water.js 9 times, palette.js 11, lanterns.js 7. It was loop between exactly 2 errors: Export 'update' is not defined in module. 53 times. Duplicate export of 'createPetals'. 10 times. Add the alias and you get the duplicate. Remove it and the export goes missing. In the last 70 steps it flipped between those 2 states 9 times. Twice it deleted the file and wrote it back identical. 85 writes against 7 reads. It almost never looked at what was already on disk. It compacted its own context 4 times and came back to the same 2 errors every time. It stopped with 3 files still failing node --check, all on the same line, update as updatePetals. 2 modules that main.js imports do not exist at all, and one of them is the lanterns file it had written 7 times. Then I gave the same model the same prompt as a single HTML file. It shipped a complete working scene in 1 pass. So it is not a capability wall. The module code it wrote is decent. It cannot hold its own module graph together, and it has nothing that tells it that it is going in circles. 438,737 output tokens and it never once said it was stuck. I spent 3 days making this thing run on my rig. An FP8 KV cache path that vLLM rejects on Ampere in 4 separate places. The 51B n-gram table baked down to FP8 so it fits in host RAM. W4A16 weights, the full 262K context, on 4 gaming GPUs from 2020. All of that works. And then it cannot wire 16 files together. So I'm going back to Qwen3.8 27B as my daily driver until something changes. Could be something in my own build doing this, I'm rebuilding the quant to find out. Either way I'll post what comes back.

We compressed Hy4-preview from 1.5TB to ～200GiB GGUF and it still works well ! Meet MIX-STQ1_0.The trick isn’t just going low, it’s deciding where: calibration data picks each layer’s bit-width, some down to 1.31-bit STQ1_0, some up to 2.06-bit IQ2_XXS. Same budget, lower error. Accuracy barely moves vs BF16 📊 MCP Atlas 83.7→83.2 📊 SWE-Bench multi 82.9→81.3 📊 MRCR 81.3→81.1 📊 IFBench 73.5→72.5 See the details on HF : AngelSlim/Hy4-preview-GGUF Weights & low-bit GGUFs 👇 #LLM# #Quantization# #llamacpp# #Hy#

Qwen3.8 Flash Next just served a 262k token prompt with an image on my 4x3090. Decode at that depth: 67 tok/s. At 260K its 66.3, so the curve is basically FLAT. Almost every number you see for this model is 20-21 tok/s on a single 24GB card with experts offloaded to RAM on llamacpp. I went the other way: vLLM, all experts on GPU, FP8 KV. Full recipe below in my github, one command, raw runs included. 🧵A deep-dive technical thread. Let's go!