TL;DR: Cortex bakes the gains an LLM gets from context into its weights, so they persist after the context is removed. After 50 sequential factual edits it recovers 88.9% of the gain while keeping 99.8% of original performance. Title: Cortex: Consolidating Context into Parametric Memory URL: Key points ・🧠 Inspired by hippocampus-to-neocortex consolidation; unifies knowledge editing, skill learning, behavior steering, and suppression as one problem ・🔮 The model generates queries anticipating how the context will be used and learns from them ・🧩 Updates only the FFN: low-rank updates (Synaptic) or new added channels (Structural), both mergeable into the weights ・👩‍🏫 A frozen teacher that sees the context distills its outputs and hidden states into a student that doesn't ・📊 Tested on 4B-120B dense and MoE models; batch editing hits 80-91% CR vs. 36-46% for the strongest baselines ・🤖 On ALFWorld, the consolidated model even beats supplying the context directly ・⚠️ Knowledge suppression still harms neighboring knowledge 💡 A practical path beyond RAG and prompting: folding experience into the weights. #LLM# #ContinualLearning#