Ash Hart(@ashxhart):I've taken matters into my own hands with @Apple's Metal. 🔥 I'm building my own custom Metal kernel for TensorFold, starting with 4-bit inference on M5. It'll set me back a little while I get this built and tested properly, but what I've achieved locally with (mlx_tf) already feels like a real breakthrough. The M3 Ultra has 819 GB/s of memory bandwidth. The M5 Ultra has 1.2 TB/s. I look at that hardware and think there's still a lot more decode performance to unlock. Why should the Sparks have all the fun... Watch this space 👀

2026.11.05 23:39

I've taken matters into my own hands with @Apple's Metal. 🔥 I'm building my own custom Metal kernel for TensorFold, starting with 4-bit inference on M5. It'll set me back a little while I get this built and tested properly, but what I've achieved locally with (mlx_tf) already feels like a real breakthrough. The M3 Ultra has 819 GB/s of memory bandwidth. The M5 Ultra has 1.2 TB/s. I look at that hardware and think there's still a lot more decode performance to unlock. Why should the Sparks have all the fun... Watch this space 👀