Ash Hart(@ashxhart):🤯 GLM-5.3-Flash 4bit on two M5 Ultras (tp=2) with TensorFold and 4x TB5 cables bonded. 🚀 114 tks decode on Prose single stream, and ~1600tks prefil. C=8 decode up to 200tps. There is headroom here as well, but this is where we landed today. A new-session TTFT with our prefix-cache lean mode is 0.06s. @digitalix Thanks for letting me use the M5Us!

Ash Hart

@ashxhart

Doing my bit to bring local AI to everyone | building | | Thoughts are my own.

Joined October 2014

480 Following 13.5K Followers

Ash Hart@ashxhart

2026.10.07 19:49

🤯 GLM-5.3-Flash 4bit on two M5 Ultras (tp=2) with TensorFold and 4x TB5 cables bonded. 🚀 114 tks decode on Prose single stream, and ~1600tks prefil. C=8 decode up to 200tps. There is headroom here as well, but this is where we landed today. A new-session TTFT with our prefix-cache lean mode is 0.06s. @digitalix Thanks for letting me use the M5Us!