Register and share your invite link to earn from video plays and referrals.

Tom Greenwald
@tomgreenwald
Building @usemagnitude - the fastest inference engine for local AI. Big fan of open source + dev tools. YC alum
Joined November 2023
289 Following    3.9K Followers
Do NOT buy a 512GB Mac Studio ‼️ Two 256GB Mac Studio M5 Ultras are ~$700 cheaper and have ~1.7x faster prefill and decode Price: $22,299 vs $21,598 Both are the 36-core CPU / 80-core GPU You can connect 2x 256GB with one Thunderbolt 5 cable + RDMA, which is fast enough to run one model across both Prefill (compute) - 1x 512 does ~135 TFLOPS - 2x 256 does ~215-245 TFLOPS - Apple claims up to 3x for a 4-Mac cluster, which works out to ~1.7x for two Decode (bandwidth) - 1x 512 has 1.2 TB/s - 2x 256 gets ~1.9-2.2 TB/s effective - With tensor parallelism each box reads its half of the weights at the same time, which works out to ~1.7x for two GLM-5.3 goes from ~33-40 tok/s to ~55-70 DeepSeek V4.1 Flash goes from ~45-60 tok/s to ~75-100 Both are 4-bit with spec decode on, estimated from M3 and M5 Ultra benchmarks If you want Qwen3.8-Flash-Next or GLM-5.3-Flash, a single 256 already runs them well and you don’t need either setup. The 512GB question is about DeepSeek V4.1 Flash (552B) and GLM-5.3 (753B), tied as the best open models on Artificial Analysis v5. They need 310-420GB at 4-bit, so neither fits on one 256
Show more