Jasper Dekoninck(@j_dekoninck):We are releasing the latest version of BrokenArXiv and ArXivMath! These benchmarks now focus on conjectures that were refuted in the last month on ArXiv, and models are executed within a harness instead of directly via API. Performance remains impressive, with GPT-6 Astra on top

Jasper Dekoninck

@j_dekoninck

PhD Student at ETH Zurich, working on mathematical evaluation of LLMs (MathArena, IMproofBench, BrokenMath, ...)

加入 June 2023

136 正在關注 4.3K 粉絲

Jasper Dekoninck@j_dekoninck

2026.09.16 11:11

We are releasing the latest version of BrokenArXiv and ArXivMath! These benchmarks now focus on conjectures that were refuted in the last month on ArXiv, and models are executed within a harness instead of directly via API. Performance remains impressive, with GPT-6 Astra on top