Jasper Dekoninck(@j_dekoninck):Unfortunately, we had to remove this entry: the authors ran this as an autoformalization task with access to source papers. Our pipeline run the models without such access. 80% is still impressive for open models on this challenging task, just not comparable to the other scores

Jasper Dekoninck

@j_dekoninck

PhD Student at ETH Zurich, working on mathematical evaluation of LLMs (MathArena, IMproofBench, BrokenMath, ...)

Joined June 2023

136 Following 4.4K Followers

Jasper Dekoninck@j_dekoninck

2026.09.17 19:10

Unfortunately, we had to remove this entry: the authors ran this as an autoformalization task with access to source papers. Our pipeline run the models without such access. 80% is still impressive for open models on this challenging task, just not comparable to the other scores