Stuart Sul(@stuart_sul):(1/9) I'm thrilled to share the open-source release of Mixture-of-Kittens (MoK), our MoE megakernel for NVL72s! MoK fuses all mixture-of-experts communication and computation into a single, fully deterministic kernel, and powers Composer training across tens of thousands of GPUs. Joint work with @nash_c_brown, @hmwildermuth, @tmwilliamlin168, and @ellev3n11

Stuart Sul

@stuart_sul

ml @cursor_ai (now @spacexai), cs phd @Stanford, @HazyResearch

加入 July 2024

160 正在关注 4.1K 粉丝

Stuart Sul@stuart_sul

2026.08.04 16:00

(1/9) I'm thrilled to share the open-source release of Mixture-of-Kittens (MoK), our MoE megakernel for NVL72s! MoK fuses all mixture-of-experts communication and computation into a single, fully deterministic kernel, and powers Composer training across tens of thousands of GPUs. Joint work with @nash_c_brown, @hmwildermuth, @tmwilliamlin168, and @ellev3n11

显示更多

437

转发到社区

热门用户