Edison Scientific, Inc(@EdisonSci):Introducing BixBench3, our latest eval for language models in science and the first eval to measure the ability of models to produce the analyses underpinning entire biology papers from raw data. BixBench3 evaluates 13 frontier models on 20 paper-derived tasks, using datasets up to 241 GB. Each task mirrors how scientists often use agents today: setting a research objective and designing a methodological plan, then delegating implementation to the agent. The best model reproduced 48% of the requested artifacts on average. Agents are starting to approach research study-scale capabilities, but there is still plenty of work to do.

2026.08.26 16:00

Introducing BixBench3, our latest eval for language models in science and the first eval to measure the ability of models to produce the analyses underpinning entire biology papers from raw data. BixBench3 evaluates 13 frontier models on 20 paper-derived tasks, using datasets up to 241 GB. Each task mirrors how scientists often use agents today: setting a research objective and designing a methodological plan, then delegating implementation to the agent. The best model reproduced 48% of the requested artifacts on average. Agents are starting to approach research study-scale capabilities, but there is still plenty of work to do.

显示更多