가입 후 초대 링크를 공유하면 동영상 재생 및 초대 보상을 받을 수 있습니다.

Rohan Paul
@rohanpaul_ai
Compiling in real-time, the race towards AGI. 🗞️ Get my daily AI analysis newsletter to your email 👉
가입 June 2014
6.8K 팔로잉 중    161.4K 팬
Coding agents fix the symptom you show them far more often than the defect underneath. Agents are good at making a visible failure disappear, but this paper finds them much weaker at restoring the science underneath, and supplied domain knowledge does not reliably help. SWE-bench Science gives agents real defects from open scientific repositories and scores public tests they can iterate against separately from private tests they never see. The best configuration, Claude Code with Opus-5, clears 96.64% of public tests but reaches 47.90% Pass@1, which requires every hidden test to pass. So the limit looks like verification rather than knowledge: an agent that cannot check supplied science against executable evidence can anchor on the explanation instead of testing it. – arxiv. org/abs/2608.19799 Title: "SWE-bench Science: Can Coding Agents Resolve Engineering Tasks in Science?"
더 보기