Cheng(@zcbenz):Some recent awesome optimizations merged in MLX: Built-in sliding window support in fused SDPA https://t.co/1H8IYZbt0G Use mma/nax for QMV with small rows and large N/K, making batched decoding 2~3x faster https://t.co/drf7Z4M1do Dequantize fp4 into float16 directly, which makes prefilling 1.4x faster for old macs https://t.co/KJycFE7cgO Add a 2-stage nax QMM kernel that makes small prompt prefill almost 2x faster for some models https://t.co/eri2elFwJ1 1-bit affine quantization https://t.co/p9inRNwXZD Thin GEMM kernels making small-M/large-N and large-M/small-N 1.5x faster https://t.co/fcrqnm8oFy 1.4x faster fused SDPA by prefetching K/V https://t.co/3mP3nHW4jv And there are actually a lot more awesome changes that I'm too exhausted to list them all, the commit history of MLX is an interesting read for anyone who wants to make open source contributions.

2026.10.28 23:56

Some recent awesome optimizations merged in MLX: Built-in sliding window support in fused SDPA Use mma/nax for QMV with small rows and large N/K, making batched decoding 2~3x faster Dequantize fp4 into float16 directly, which makes prefilling 1.4x faster for old macs Add a 2-stage nax QMM kernel that makes small prompt prefill almost 2x faster for some models 1-bit affine quantization Thin GEMM kernels making small-M/large-N and large-M/small-N 1.5x faster 1.4x faster fused SDPA by prefetching K/V And there are actually a lot more awesome changes that I'm too exhausted to list them all, the commit history of MLX is an interesting read for anyone who wants to make open source contributions.