Co-Founder, CTO-CPO of @SentoraHQ (fmr IntoTheBlock), Co-Founder of @layerlens_ai, @neuralfabric( acq by Cisco) and The Sequence, Teaching at Columbia-Wharton
More about recursive self improvement in TheSequence today:
Your AI coding assistant learns a new API mid-conversation and uses it perfectly. Then you open a fresh session, and it's gone. 🧠💨
Showing a skill in context isn't the same as acquiring it. If we want AI systems that improve through experience, the real question is how experience should change the model, and who decides.
In today's The Sequence Knowledge, we dig into SEAL (Self-Adapting Language Models), which hands part of that decision to the model itself. It writes its own "self-edits," which are training data and update directives. It applies them, and reinforcement learning rewards whichever edits actually make the updated model better.
The result in one knowledge setting: QA accuracy rises from 32.7% to 47.0%. Training on the raw passage gets you just 33.5%. 👀
We also cover:
🔹 Remembering vs. learning: context, external memory, and weight updates
🔹 Why a training example is really a choice about computation
🔹 What LoRA solves, and what it doesn't
🔹 Nested Learning and Titans: adaptation across timescales
🔹 Catastrophic forgetting, the tax every persistent learner pays
The big idea is models that get better at becoming better.
Read it here 👇