Mia(@MiaAI_lab):Run GLM 5.3 Flash EXL3 with TensorFold ⚡️ Now on 4x DGX Sparks, switchless or not! For now, TP4 is on its own branch, so please make sure you use the correct branch! - Around 6M KV cache. - Decode on prose 95 tok/s, single stream - Decode on prose 163 tok/s, 4 streams - Decode on prose 222 tok/s, 8 streams - Prefill ~2500 tok/s across the board. These numbers were measured on a switchless setup! The recipe works both with or without a switch, each has it's own startup script. Links below 👇

Mia

@MiaAI_lab

Building with AI & LLMs // Running frontier open models locally

加入 July 2022

359 正在關注 41.8K 粉絲

Mia@MiaAI_lab

2026.10.10 15:16

Run GLM 5.3 Flash EXL3 with TensorFold ⚡️ Now on 4x DGX Sparks, switchless or not! For now, TP4 is on its own branch, so please make sure you use the correct branch! - Around 6M KV cache. - Decode on prose 95 tok/s, single stream - Decode on prose 163 tok/s, 4 streams - Decode on prose 222 tok/s, 8 streams - Prefill ~2500 tok/s across the board. These numbers were measured on a switchless setup! The recipe works both with or without a switch, each has it's own startup script. Links below 👇