Andrew White 🐦‍⬛(@andrewwhite01):Bixbench3: Can models reproduce the entire analysis from papers? We had agents repro papers end-to-end, with over 1B tokens spent on some. Frontier agents still score below 50%, and due to real problems like environment misconfig, quitting, or faking data 1/6

Andrew White 🐦‍⬛

@andrewwhite01

Automating science. Cofounder @EdisonSci. Cofounder @FutureHouseSF. Former prof of chem eng.

参加 April 2009

1.9K フォロー中 33.7K ファン

Andrew White 🐦‍⬛@andrewwhite01

2026.08.26 16:29

Bixbench3: Can models reproduce the entire analysis from papers? We had agents repro papers end-to-end, with over 1B tokens spent on some. Frontier agents still score below 50%, and due to real problems like environment misconfig, quitting, or faking data 1/6