登録して招待リンクを共有すると、動画再生報酬と紹介報酬を獲得できます。

Tom Greenwald
@tomgreenwald
Building @usemagnitude. Coding agents, dev tools, open source. YC S25
参加 November 2023
287 フォロー中    3.4K ファン
We compared the M5 Ultra Mac Studio 512GB with 4x DGX Spark and 4x AMD Ryzen AI Halo The prices are relatively the same ($15-20k), but the trade-offs are noticeable Compute (prompt processing speed) - Mac: one chip, all the compute is always usable. Quick on normal prompts, will feel slower on long ones - Sparks: boxes link over 200GbE, which is fast enough to combine their compute. Fastest of the three, and native FP4 speeds up quantized models even more - Halos: boxes link over 10GbE, too slow to share work properly. Ends up close to the Mac but somewhat worse Bandwidth (token generation speed) - Mac: 512GB on a single bus, faster than a 4090, ~4x the tokens/sec of Spark or Halos - Sparks: the model splits across boxes and tokens pass through them in sequence, so 4 boxes still generate at 1 box's speed - Halos: same ceiling as Spark, same reason Power - Mac: less than a gaming PC, silent - Sparks: nearly maxes out a wall circuit, will be hot - Halos: about half the Sparks, but somewhat hot The real gap is bandwidth. The Studio will feel noticeably faster. Spark 2 is rumored soon, but it won't matter unless the bandwidth goes way up. Same for AMD, plus better interconnect speeds
もっと見る