็™ป้Œฒใ—ใฆๆ‹›ๅพ…ใƒชใƒณใ‚ฏใ‚’ๅ…ฑๆœ‰ใ™ใ‚‹ใจใ€ๅ‹•็”ปๅ†็”Ÿๅ ฑ้…ฌใจ็ดนไป‹ๅ ฑ้…ฌใ‚’็ฒๅพ—ใงใใพใ™ใ€‚

David Hendrickson
@TeksEdge
CEO & Founder | PhD | Startup Advisor | @Columbia | Author Generative Software Engineering | ๐Ÿ”” Follow for AI & Vibe Coding Tips ๐Ÿ‘‡
ๅ‚ๅŠ  July 2023
549 ใƒ•ใ‚ฉใƒญใƒผไธญ    11.2K ใƒ•ใ‚กใƒณ
๐Ÿคฏ Xiaomi showed a 150W mini AI box designed to run a 120B model LOCALLY. Yeah, it's Xiaomi, so good luck getting it in the US if it ever ships. The Xiaomi AI Cube, a prototype built around 3 of Xiaomi's own XRING chips. And the specs are kind of crazy ๐Ÿ‘‰ ๐Ÿง  Local deployment โ†’ 120B + 3B models ๐Ÿ’พ D100 โ†’ up to 160GB unified memory ๐Ÿ‘€ ๐Ÿš€ O100 โ†’ 1.22 TB/s near-memory bandwidth ๐Ÿ‘€ Nvidia did you see this? โšก Entire AI Cube โ†’ up to 150W ๐Ÿงฎ O3 โ†’ 200 TOPS NPU ๐ŸŽฎ O3 โ†’ 16-core G2 Ultra NX GPU That 1.22 TB/s number is especially interesting. According to Xiaomi, they stack high-speed DRAM directly over the O100's logic/NPU layer using wafer-on-wafer packaging + hybrid bonding. In other words ๐Ÿ‘‰ very short path between memory and compute. And Xiaomi says the D100 itself can accommodate local models as large as 200B parameters. ๐Ÿ‘€ ๐ŸŽฏ Strix Halo showed what 128GB unified memory could do. Gorgon Halo is up next with 192GB. Now Xiaomi is experimenting with 160GB-class unified memory + >1 TB/s bandwidth in a 150W mini box. Caveats of course ... โš ๏ธ It's a prototype โš ๏ธ No price โš ๏ธ No retail release date โš ๏ธ O100/D100 commercial use is planned for 2027 ๐Ÿ”— Source: GizmoChina
ใ‚‚ใฃใจ่ฆ‹ใ‚‹