Baseten(@baseten):"So, can a model learn facts continually in its weights? Creating usable knowledge: solvable Preserving capability: solvable Keeping earlier facts reachable: unsolved" New research from our Head of Training just dropped on arXiv.

2026.07.17 20:26

"So, can a model learn facts continually in its weights? Creating usable knowledge: solvable Preserving capability: solvable Keeping earlier facts reachable: unsolved" New research from our Head of Training just dropped on arXiv.

Charlie O'Neill@oneill_c

2026.07.16 17:23

1/ Can you actually get new facts into an LLM's weights without breaking the model? This question decides how we approach continual learning: should memory live in the context (retrieval, compressed caches) or in the weights themselves? We spent a long time measuring it, and it breaks somewhere much stranger than we expected, making us much more bullish on compressed kv caches and ICL for continual learning, as opposed to weight updates themselves 🧵