๊ฐ€์ž… ํ›„ ์ดˆ๋Œ€ ๋งํฌ๋ฅผ ๊ณต์œ ํ•˜๋ฉด ๋™์˜์ƒ ์žฌ์ƒ ๋ฐ ์ดˆ๋Œ€ ๋ณด์ƒ์„ ๋ฐ›์„ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

David Hendrickson
@TeksEdge
CEO & Founder | PhD | Startup Advisor | @Columbia | Author Generative Software Engineering | ๐Ÿ”” Follow for AI & Vibe Coding Tips ๐Ÿ‘‡
๊ฐ€์ž… July 2023
552 ํŒ”๋กœ์ž‰ ์ค‘    12.5K ํŒฌ
Nvidia has delivered a Qwen3.6-35B killer and it's a 74B โ†’ 24GB model, actually, today. And now we have real numbers. ๐Ÿ‘€ Nemotron 3.5 Bolt 74B-A9B ๐Ÿง  74B total / 9B active ๐Ÿ’พ 22.1GB quantized weights ๐Ÿ”ข ~2.06-bit experts + QAT ๐Ÿ“š up to 256K context ๐Ÿ‘๏ธ text + images + video ๐Ÿค– reasoning + tool use ๐Ÿฆ™ official GGUF + llama.cpp And NVIDIA reports on RTX Spark: โšก ~84 tps That's a 74B multimodal agent model running locally. Check out how much quality survived the 2-bit compression. BF16 โ†’ 24GB IQ2: ๐ŸŽฏ PinchBench 90.35 โ†’ 87.82 ๐Ÿ”Ž BrowseComp 70.0 โ†’ 66.75 ๐Ÿง  GPQA Diamond 88.83 โ†’ 86.43 ๐Ÿ’ป Terminal-Bench 2.1 53.93 โ†’ 50.98 Nvidia calculates ๐Ÿ’พ weights โ†’ 22.01GB ๐Ÿ’พ cache + activations โ†’ 0.87GB ๐Ÿ’พ LiLiCorr drafter โ†’ 0.96GB Total โ†’ 23.83GB The vision encoder/projector are not included in that number. So Nvidia's official llama.cpp recipes actually target ๐ŸŸข 32GB+ GPU โ†’ 65K context / 28GB config ๐Ÿ”ฅ 48GB+ GPU โ†’ 256K context / 32GB config ๐Ÿ”— Links in ALT
๋” ๋ณด๊ธฐ