Assaf Ben Kish(@abk_tau):(1/7) Humans can acquire many capabilities without (catastrophically) forgetting previous ones. Why can't neural networks do the same? Local Support Learning (LSL) takes us a step closer towards this goal. In LLMs of up to 7B parameters, LSL augments gradient-based training to retain prior capabilities while learning new tasks at full capacity. Notably, it does so without access to prior data. Key idea: casting forgetting as a geometric problem, and optimizing for the worst case. Paper: https://t.co/lZ6UwvQh0C Website and code:

Assaf Ben Kish

@abk_tau

Deep Learning | Large Language Models | Reinforcement Learning

参加 August 2023

199 フォロー中 298 ファン

Assaf Ben Kish@abk_tau

2026.10.02 12:56

(1/7) Humans can acquire many capabilities without (catastrophically) forgetting previous ones. Why can't neural networks do the same? Local Support Learning (LSL) takes us a step closer towards this goal. In LLMs of up to 7B parameters, LSL augments gradient-based training to retain prior capabilities while learning new tasks at full capacity. Notably, it does so without access to prior data. Key idea: casting forgetting as a geometric problem, and optimizing for the worst case. Paper: Website and code: