Jasper Dekoninck(@j_dekoninck):We are releasing the latest version of BrokenArXiv and ArXivMath! These benchmarks now focus on conjectures that were refuted in the last month on ArXiv, and models are executed within a harness instead of directly via API. Performance remains impressive, with GPT-6 Astra on top

Jasper Dekoninck

@j_dekoninck

PhD Student at ETH Zurich, working on mathematical evaluation of LLMs (MathArena, IMproofBench, BrokenMath, ...)

Joined June 2023

136 Following 4.3K Followers

Jasper Dekoninck@j_dekoninck

2026.09.16 11:11

We are releasing the latest version of BrokenArXiv and ArXivMath! These benchmarks now focus on conjectures that were refuted in the last month on ArXiv, and models are executed within a harness instead of directly via API. Performance remains impressive, with GPT-6 Astra on top