Nvidia has delivered a Qwen3.6-35B killer and it's a 74B โ 24GB model, actually, today.
And now we have real numbers. ๐
Nemotron 3.5 Bolt 74B-A9B
๐ง 74B total / 9B active
๐พ 22.1GB quantized weights
๐ข ~2.06-bit experts + QAT
๐ up to 256K context
๐๏ธ text + images + video
๐ค reasoning + tool use
๐ฆ official GGUF + llama.cpp
And NVIDIA reports on RTX Spark:
โก ~84 tps
That's a 74B multimodal agent model running locally.
Check out how much quality survived the 2-bit compression. BF16 โ 24GB IQ2:
๐ฏ PinchBench
90.35 โ 87.82
๐ BrowseComp
70.0 โ 66.75
๐ง GPQA Diamond
88.83 โ 86.43
๐ป Terminal-Bench 2.1
53.93 โ 50.98
Nvidia calculates
๐พ weights โ 22.01GB
๐พ cache + activations โ 0.87GB
๐พ LiLiCorr drafter โ 0.96GB
Total โ 23.83GB
The vision encoder/projector are not included in that number.
So Nvidia's official llama.cpp recipes actually target
๐ข 32GB+ GPU โ 65K context / 28GB config
๐ฅ 48GB+ GPU โ 256K context / 32GB config
๐ Links in ALT