Stuart Sul
@stuart_sul
160 Following 4.1K Followers
(1/9) I'm thrilled to share the open-source release of Mixture-of-Kittens (MoK), our MoE megakernel for NVL72s! MoK fuses all mixture-of-experts communication and computation into a single, fully deterministic kernel, and powers Composer training across tens of thousands of GPUs.
Joint work with @nash_c_brown, @hmwildermuth, @tmwilliamlin168, and @ellev3n11
Show more
