Today we introduce Foresight-SWE 🦉: a benchmark that tests whether coding agents can catch their own bugs.
tl;dr: They don't!!
Across 11 models, coding agents solve developer-specified issues 60.7% of the time on average, but only 14.8% when they must additionally avoid bugs developers later fixed.
Show more
