Nvidia has delivered a Qwen3.6-35B killer and it's a 74B â 24GB model, actually, today.
And now we have real numbers. đ
Nemotron 3.5 Bolt 74B-A9B
đ§ 74B total / 9B active
đž 22.1GB quantized weights
đĸ ~2.06-bit experts + QAT
đ up to 256K context
đī¸ text + images + video
đ¤ reasoning + tool use
đĻ official GGUF + llama.cpp
And NVIDIA reports on RTX Spark:
⥠~84 tps
That's a 74B multimodal agent model running locally.
Check out how much quality survived the 2-bit compression. BF16 â 24GB IQ2:
đ¯ PinchBench
90.35 â 87.82
đ BrowseComp
70.0 â 66.75
đ§ GPQA Diamond
88.83 â 86.43
đģ Terminal-Bench 2.1
53.93 â 50.98
Nvidia calculates
đž weights â 22.01GB
đž cache + activations â 0.87GB
đž LiLiCorr drafter â 0.96GB
Total â 23.83GB
The vision encoder/projector are not included in that number.
So Nvidia's official llama.cpp recipes actually target
đĸ 32GB+ GPU â 65K context / 28GB config
đĨ 48GB+ GPU â 256K context / 32GB config
đ Links in ALT