alphaXiv(@askalphaxiv):"Matryoshka Language Model Suites" Instead of training every model size separately, this paper nests 500M, 1.5B, and 3B models inside one architecture and trains them together. The smaller models are standalone checkpoints, get near-free distillation from the largest model, and share weights + KV cache for speculative decoding. And you still get the same performance, with 36% less training compute, and 14-26% faster speculative decoding. So a model family can become one jointly trained system instead of several independent models.

2026.08.21 00:20

"Matryoshka Language Model Suites" Instead of training every model size separately, this paper nests 500M, 1.5B, and 3B models inside one architecture and trains them together. The smaller models are standalone checkpoints, get near-free distillation from the largest model, and share weights + KV cache for speculative decoding. And you still get the same performance, with 36% less training compute, and 14-26% faster speculative decoding. So a model family can become one jointly trained system instead of several independent models.