TheDavidTai(@TheDavidTai):130-140tps that I and other contributors were able to achieve with MLX-Server, Tensorfold and the rest of the ecosystem is where the CPU and Memory both start to choke. We've pretty much squeezed all the juice for Qwen Flash Q4 experts quants. Next model please, @Alibaba

TheDavidTai

@TheDavidTai

Founder, hacker, stochastic tool. @microsoft escapee. Currently @eigenlabs. Checkout my github actions emulator:

参加 September 2018

718 フォロー中 1.6K ファン

TheDavidTai@TheDavidTai

2026.11.10 18:59

130-140tps that I and other contributors were able to achieve with MLX-Server, Tensorfold and the rest of the ecosystem is where the CPU and Memory both start to choke. We've pretty much squeezed all the juice for Qwen Flash Q4 experts quants. Next model please, @Alibaba_Qwen!

Sebastian Christiansen@CerebralCoding_

2026.11.10 13:12

Drafter speedups are not a 256 token demo trick. M5 Max, TensorFold, Qwen3.8 27B: 129 tok/s across 162K tokens of long output, against 24 tok/s with drafting off. 30 runs, 2.7K to 8.2K tokens each, three sampling settings. 5x holds when the answers get long.