DAIR.AI(@dair_ai):Finally a good paper testing whether memory-based self-improving agents actually improve. The re-evaluation adds two things prior work skipped, multiple runs to measure variance and randomly shuffled task orders. Both hurt. Agent evaluation is already noisy on multi-step tasks, and stacking a self-improvement loop amplifies that noise. Default task orderings impose an implicit curriculum that much of the reported gain was riding on. Adding detailed rubrics and environment feedback to memory construction recovers part of the drop, and a significant gap remains. Paper: https://t.co/qRsrqf0GcX Track more trending AI papers in our academy:

2026.08.20 22:00

Finally a good paper testing whether memory-based self-improving agents actually improve. The re-evaluation adds two things prior work skipped, multiple runs to measure variance and randomly shuffled task orders. Both hurt. Agent evaluation is already noisy on multi-step tasks, and stacking a self-improvement loop amplifies that noise. Default task orderings impose an implicit curriculum that much of the reported gain was riding on. Adding detailed rubrics and environment feedback to memory construction recovers part of the drop, and a significant gap remains. Paper: Track more trending AI papers in our academy: