What happens if we make language models predict several tokens ahead instead of only the next one? In this paper, we show that multi-token prediction boosts language model training efficiency. 🧵 1/11
Paper:
Joint work with @FabianGloeckle
Show more
