注册并分享邀请链接,可获得视频播放与邀请奖励。

Tom Greenwald
@tomgreenwald
Building @usemagnitude. Coding agents, dev tools, open source. YC S25
加入 November 2023
287 正在关注    3.4K 粉丝
We compared the M5 Ultra Mac Studio 512GB with 4x DGX Spark and 4x AMD Ryzen AI Halo The prices are relatively the same ($15-20k), but the trade-offs are noticeable Compute (prompt processing speed) - Mac: one chip, all the compute is always usable. Quick on normal prompts, will feel slower on long ones - Sparks: boxes link over 200GbE, which is fast enough to combine their compute. Fastest of the three, and native FP4 speeds up quantized models even more - Halos: boxes link over 10GbE, too slow to share work properly. Ends up close to the Mac but somewhat worse Bandwidth (token generation speed) - Mac: 512GB on a single bus, faster than a 4090, ~4x the tokens/sec of Spark or Halos - Sparks: the model splits across boxes and tokens pass through them in sequence, so 4 boxes still generate at 1 box's speed - Halos: same ceiling as Spark, same reason Power - Mac: less than a gaming PC, silent - Sparks: nearly maxes out a wall circuit, will be hot - Halos: about half the Sparks, but somewhat hot The real gap is bandwidth. The Studio will feel noticeably faster. Spark 2 is rumored soon, but it won't matter unless the bandwidth goes way up. Same for AMD, plus better interconnect speeds
显示更多
0
73
763
76
转发到社区