๊ฐ€์ž… ํ›„ ์ดˆ๋Œ€ ๋งํฌ๋ฅผ ๊ณต์œ ํ•˜๋ฉด ๋™์˜์ƒ ์žฌ์ƒ ๋ฐ ์ดˆ๋Œ€ ๋ณด์ƒ์„ ๋ฐ›์„ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

David Hendrickson
@TeksEdge
CEO & Founder | PhD | Startup Advisor | @Columbia | Author Generative Software Engineering | ๐Ÿ”” Follow for AI & Vibe Coding Tips ๐Ÿ‘‡
๊ฐ€์ž… July 2023
551 ํŒ”๋กœ์ž‰ ์ค‘    12.3K ํŒฌ
2 x ~$400 Snapdragon mini PCs. No GPU used and 64GB of RAM combined. And they're running a 95GB Qwen3.8-Flash-Next at 20.3 tps. ๐Ÿ‘€ The model is 31GB larger than their combined RAM. I have to build this. The setup ๐Ÿ–ฅ๏ธ 2ร— Lenovo IdeaCentre Mini โš™๏ธ Snapdragon X X1-26-100 ๐Ÿ’พ 32GB LPDDR5 each ๐Ÿ’ฝ 512GB NVMe each ๐Ÿ”Œ connected with one USB4 cable ๐Ÿ’ฐ ~$800 total Here's the model: ๐Ÿง  Qwen3.8-Flash-Next ๐Ÿ“ฆ IQ4_NL โ†’ 95GB Reported with the author's ToRc inference engine ๐Ÿ“ฅ Read โ†’ 46.2 tps โœ๏ธ Generation โ†’ 20.3 tps ๐Ÿ’ป Coding generation โ†’ 19.2 tps And the Adreno GPUs aren't being used. ToRc uses the Snapdragon CPUs + Hexagon NPUs for some of the dense math, while the model's experts are streamed from NVMe as they're needed because the weights don't fit in RAM. The second box also isn't simply splitting normal token generation in half. For Flash-Next, ToRc pipelines MTP across the two machines over USB4. โš ๏ธ ToRc isn't publicly released yet and these are the author's results, not independently reproduced benchmarks. Link in ALT
๋” ๋ณด๊ธฐ