Alex Ziskind
@digitalix
Software engineer making videos about dev gear, local AI, and weird tech experiments
加入 March 2009
1.5K 正在關注 28.7K 粉絲
crazy numbers for GLM on dual-M5 Ultras
🤯 GLM-5.3-Flash 4bit on two M5 Ultras (tp=2) with TensorFold and 4x TB5 cables bonded. 🚀
114 tks decode on Prose single stream, and ~1600tks prefil. C=8 decode up to 200tps.
There is headroom here as well, but this is where we landed today.
A new-session TTFT with our prefix-cache lean mode is 0.06s.
@digitalix Thanks for letting me use the M5Us!
顯示更多
