We are integrating Mixture-of-kittens into Nvidia NeMo-Automodel. 7%-50% e2e bf16 training speed gain on gb200 from ep4 to ep64 settings. mxfp8 integration coming soon!
Checkout our PR for experiment details:
And Zhihu tutorial:
Show more
