註冊並分享邀請連結,可獲得影片播放與邀請獎勵。

Cheng
@zcbenz
maintainer of MLX @apple. creator of @electronjs. check for the open source things I built.
加入 June 2007
108 正在關注    7.8K 粉絲
Some recent awesome optimizations merged in MLX: Built-in sliding window support in fused SDPA Use mma/nax for QMV with small rows and large N/K, making batched decoding 2~3x faster Dequantize fp4 into float16 directly, which makes prefilling 1.4x faster for old macs Add a 2-stage nax QMM kernel that makes small prompt prefill almost 2x faster for some models 1-bit affine quantization Thin GEMM kernels making small-M/large-N and large-M/small-N 1.5x faster 1.4x faster fused SDPA by prefetching K/V And there are actually a lot more awesome changes that I'm too exhausted to list them all, the commit history of MLX is an interesting read for anyone who wants to make open source contributions.
顯示更多
0
14
163
15
轉發到社區