A 4-bit model has not lost a single parameter. Every one is still there, stored in fewer bits. The card has the arithmetic. Here is what it does not show. A dense model decoding at small batch size spends every step streaming its entire weight set through the chip and doing very little work per weight. That is what weight-only quantization attacks, which is why the gain lands in decode and shrinks in prefill, where one weight read already serves many positions. The error is the interesting part. A small weight can take a large relative rounding error, and one outlier can stretch a group's scale and cost every other weight in it some precision. But what decides whether any of that matters is the activation the weight gets multiplied by. That is why the methods that work look at activation statistics, not just weight magnitudes. Two things to watch. Activation quantization, because recent accelerators support FP8 and FP4 natively, and once both operands are low precision the arithmetic gets cheaper too. And KV cache quantization, because longer contexts grow both the cache and the traffic to read it every step. Fewer bits per weight. Not fewer weights.