Run Kimi K3 on a Mac Studio 🫰
K3 is 2.8T parameters and 1.6TB on disk, which makes it impossible to run on Apple Silicon.
Until now.
Our MLX port is now open source:
To accomplish this, we solved two things:
1. We wrote a streaming converter that walks one layer at a time, so that mlx_lm doesn't need to materialize the whole model.
2. REAP pruning sits on top and scores all 896 experts against a calibration corpus to keep only ones your workload needs.
That's what brings K3 down to 350GB and inside a Mac Studio.
Show more
