51B N-gram embeddings is a massive trainable lookup-memory for local token combinations. The model takes the last few tokens, calculates the hash of their N-grams, fetches a few vectors from tables, and mixes them into the hidden state. So the main model spends less effort reconstructing frequent local patterns. Each token uses a tiny fraction of the lookup table, but all 51B parameters still need to be stored somewhere - in VRAM, RAM, or via a dedicated offload. At ideal 4 bpw, the math goes like this: 125B main weights - 58.2 GiB 51B N-gram tables - 23.7 GiB total - 82.0 GiB On my 4x3090s, about 14 GiB will be left over. But that's the best-case scenario. Real Q4 is usually heavier than four bits per parameter, and some quantizers leave embedding tables in FP16 or BF16 altogether. So the main question of the release isn't just the model size. We need to see the dtype of these 51B, the actual checkpoint, and support for quantization or offloading. This exact N-gram bastard is gonna decide whether I run Qwen4 architecture at home or not.