Nathan Godey(@nthngdy):🧵New paper: Matryoshka LM Suites Instead of training all models of a suite independently, we nest them within a single architecture and train the suite in a single run => much less compute, same performance Also allows free distillation, better speculative decoding & more 👇

Nathan Godey

@nthngdy

Post-doc at Cornell Tech

Joined November 2021

856 Following 1.5K Followers

Nathan Godey@nthngdy

2026.08.19 13:47

🧵New paper: Matryoshka LM Suites Instead of training all models of a suite independently, we nest them within a single architecture and train the suite in a single run => much less compute, same performance Also allows free distillation, better speculative decoding & more 👇