The question that is stopping me from buying an M5 Ultra: why does a Mac with 1.2 TB/s of bandwidth lose to DGX Spark at long context and with multiple users?
With one user, decode is memory-bound. Every token means reading all the active weights once, and the GPU mostly waits for data. More bandwidth means more tok/s, and that's where the Mac shines. 👍
Batching changes everything. With 8 users you read the weights once and use them for 8 tokens. Same bytes, 8× the math. Throughput scales almost for free… until compute becomes the bottleneck. 👀
That's where the Mac hits a wall: good bandwidth, lower low-precision compute. The Spark's Blackwell tensor cores with native FP4 give far more FLOPs per GB/s of bandwidth, so it keeps scaling while the Mac stops.
Real numbers on DeepSeek V4.1 Flash with TensorFold, total tok/s from 1 to 8 users (chart below):
2× Spark: 77 → 105 → 150 → 194 (
@rwojo)
1× M5 Ultra: 63 → 78 → 98 → 99 (
@TechMDAI, first runs, capped at 4 slots)
Long context is even worse. Prefill is pure compute, attention grows with context length, and the KV cache is per-user: no way to share it. On DeepSeek V4 Flash the Mac's decode drops from ~60 to ~27 tok/s, while 2 Sparks hold ~51 (tests by
@satsuki2694).
Bottom line: the Mac is unbeatable for fitting huge models (MoE) in a single box, but the Spark wins when you need compute: long prompts, many users, agents. Different tools, different jobs.
I run both, and I was planning a deep dive on the M5 Ultra to really understand its limits, but at this price I don't know if it makes sense spending all these money just to make tests and share data. I'll try to figure out something, otherwise I'll wait for others to test these concurrency limits in depth. 💪