in the next 2 minutes, I'll walk you through why every single AI company, including Nvidia, was unnecessarily sacrificing model intelligence, speed, and often both... and how we fixed it the tldr: - error(layer a) + error(layer b) < error(layer a) alone - quantizing MORE of the model can result in the same or HIGHER quality model, if you know which layers to quantize. let's take GLM5.2 architecture as an example. tokens here go through: 1) normalize 2) attention 3) normalize 4) moe repeated 78 times moe: > just a big mlp, split into 257 separate smaller mlps (experts) > 8 are routed and one is shared (only 3.5% of the mlp is activated) so we say, let's make the matrix multiplications faster. we quantize the matrices to nvfp4 but what do you quantize: every single one of the 257 experts, the attention projections, both? Nvidia, and every other quant scheme, settles on a somewhat intuitive solution: """ a) there is only one router per layer, only one shared expert, and only one output projection per layer. b) every single token goes through these, so they must be pretty important, and if we quantize these, errors will compound across the xN number of tokens input. c) so let's leave the shared experts and projections unquantized. d) let's quantize ONLY the 256 routed experts """ -this is literally nvidia/GLM-5.2-NVFP4 intuitive. simple to reason about. wrong. empirically: if you have a model that processes its input through layer a, then layer b, it is possible that a scheme quantizing BOTH layer a and layer b performs better than a scheme quantizing just one of these. by way of example, our config quantizes every single out projection (all 78) of the attention, and 43 of the 78 shared experts. Nvidia quantizes none of these. we score the same on benchmarks (quality). we get a free 20% higher throughput. i leave the math proof as an exercise for the reader in the paper... and some pictures to click through if you'd rather skip the (43 pages of) math