Just wow. Hats off to Apple. 🫡 Apple ran an 816B open-weight model locally at 36.7 tok/s across four Mac Studios. They connected the Macs with Thunderbolt 5 cables. Apple used the new M5 Ultra Mac Studios + MLX to run Muse Spark Open: 🧠 816B total / 42B active 💾 NVFP4 → ~493GB of weights 🍎 4× M5 Ultra Mac Studios 🔗 RDMA over Thunderbolt 5 📚 8K context Performance: 🍎🍎 2 Macs, pipeline → 15.74 tps 🍎🍎 2 Macs, tensor parallel → 24.99 tps 🍎🍎🍎🍎 4 Macs, tensor parallel → 36.73 tps And on four Macs: ⚡ Time to first token → 4.8 sec Muse Spark Open is 816B parameters. Even quantized to NVFP4, the weights alone occupy about 493GB. A maxed-out Mac Studio has 512GB unified memory, but once you add the memory required to actually run inference, the model no longer fits on one. So Apple splits the model across multiple Macs and uses RDMA over Thunderbolt 5 to make them behave like one local inference system. Each pair of Macs is connected peer-to-peer with two Thunderbolt cables. 🔗 Apple MLX research link in ALT.