Przemek Chojecki | PC(@prz_chojecki):We're testing our new benchmark. STEMBench is a set of 120 research-level problems from 6 domains - Biology, Chemistry, Physics, Astronomy, Earth science, Engineering - 20 problems per domain. Very cool to see that our initial calibration works well and the benchmark is far from saturated (max score is 4 per domain). Detailed testing with a dedicated website incoming. Interestingly, it seems like physics will be next after mathematics. Math is not included in this benchmark, we have ErdosBench for that. DM me if you want to see your model tested.

Przemek Chojecki | PC

@prz_chojecki

Research-level Math Data + hard STEM & coding RL tasks for LLMs @ PhD in mathematics.

参加 April 2016

373 フォロー中 15.3K ファン

Przemek Chojecki | PC@prz_chojecki

2026.10.06 09:34

We're testing our new benchmark. STEMBench is a set of 120 research-level problems from 6 domains - Biology, Chemistry, Physics, Astronomy, Earth science, Engineering - 20 problems per domain. Very cool to see that our initial calibration works well and the benchmark is far from saturated (max score is 4 per domain). Detailed testing with a dedicated website incoming. Interestingly, it seems like physics will be next after mathematics. Math is not included in this benchmark, we have ErdosBench for that. DM me if you want to see your model tested.