We compared the M5 Ultra Mac Studio 512GB with 4x DGX Spark and 4x AMD Ryzen AI Halo
The prices are relatively the same ($15-20k), but the trade-offs are noticeable
Compute (prompt processing speed)
- Mac: one chip, all the compute is always usable. Quick on normal prompts, will feel slower on long ones
- Sparks: boxes link over 200GbE, which is fast enough to combine their compute. Fastest of the three, and native FP4 speeds up quantized models even more
- Halos: boxes link over 10GbE, too slow to share work properly. Ends up close to the Mac but somewhat worse
Bandwidth (token generation speed)
- Mac: 512GB on a single bus, faster than a 4090, ~4x the tokens/sec of Spark or Halos
- Sparks: the model splits across boxes and tokens pass through them in sequence, so 4 boxes still generate at 1 box's speed
- Halos: same ceiling as Spark, same reason
Power
- Mac: less than a gaming PC, silent
- Sparks: nearly maxes out a wall circuit, will be hot
- Halos: about half the Sparks, but somewhat hot
The real gap is bandwidth. The Studio will feel noticeably faster. Spark 2 is rumored soon, but it won't matter unless the bandwidth goes way up. Same for AMD, plus better interconnect speeds