Jessy Lin(@realJessyLin):Building a good benchmark for continual learning takes a lot of thought -- it's non-trivial to make it hard in the ways that matter. excited to see people working towards this

2026.05.04 20:25

Building a good benchmark for continual learning takes a lot of thought -- it's non-trivial to make it hard in the ways that matter. excited to see people working towards this

Parth Asawa@pgasawa

2026.05.04 17:59

Today, we’re releasing Continual Learning Bench 1.0: the first, realistic benchmark for measuring how AI systems can improve in online settings. Benchmarks today assume models are stateless. Each example is independent, and once a system finishes a task, it moves on as if nothing happened. But deployed AI systems should learn from experience. We tested 10+ frontier systems against novel, expert-validated tasks and find there’s still plenty of headroom for learning. (1/n)