註冊並分享邀請連結,可獲得影片播放與邀請獎勵。

Przemek Chojecki | PC
@prz_chojecki
Research-level Math Data + hard STEM & coding RL tasks for LLMs @ PhD in mathematics.
加入 April 2016
373 正在關注    15.3K 粉絲
We're testing our new benchmark. STEMBench is a set of 120 research-level problems from 6 domains - Biology, Chemistry, Physics, Astronomy, Earth science, Engineering - 20 problems per domain. Very cool to see that our initial calibration works well and the benchmark is far from saturated (max score is 4 per domain). Detailed testing with a dedicated website incoming. Interestingly, it seems like physics will be next after mathematics. Math is not included in this benchmark, we have ErdosBench for that. DM me if you want to see your model tested.
顯示更多