Przemek Chojecki | PC(@prz_chojecki):We're testing our new benchmark. STEMBench is a set of 120 research-level problems from 6 domains - Biology, Chemistry, Physics, Astronomy, Earth science, Engineering - 20 problems per domain. Very cool to see that our initial calibration works well and the benchmark is far from saturated (max score is 4 per domain). Detailed testing with a dedicated website incoming. Interestingly, it seems like physics will be next after mathematics. Math is not included in this benchmark, we have ErdosBench for that. DM me if you want to see your model tested.

2026.10.06 09:34

We're testing our new benchmark. STEMBench is a set of 120 research-level problems from 6 domains - Biology, Chemistry, Physics, Astronomy, Earth science, Engineering - 20 problems per domain. Very cool to see that our initial calibration works well and the benchmark is far from saturated (max score is 4 per domain). Detailed testing with a dedicated website incoming. Interestingly, it seems like physics will be next after mathematics. Math is not included in this benchmark, we have ErdosBench for that. DM me if you want to see your model tested.