注册并分享邀请链接,可获得视频播放与邀请奖励。

Pipe Network
@pipenetwork
Pipe Network AI Labs
加入 April 2022
27 正在关注    176.2K 粉丝
Run Kimi K3 on a Mac Studio 🫰 K3 is 2.8T parameters and 1.6TB on disk, which makes it impossible to run on Apple Silicon. Until now. Our MLX port is now open source: To accomplish this, we solved two things: 1. We wrote a streaming converter that walks one layer at a time, so that mlx_lm doesn't need to materialize the whole model. 2. REAP pruning sits on top and scores all 896 experts against a calibration corpus to keep only ones your workload needs. That's what brings K3 down to 350GB and inside a Mac Studio.
显示更多
0
123
3K
319
转发到社区