Chubby♨️(@kimmonismus):P-Zero Research's new TasteVal benchmark measures research taste: how well a model designs experiments and reads their results on eight real AI-development tasks, scored by how much compute it needs compared with human experts (the dashed 1x line). Progress was slow from GPT-4 in 2023 through 2025, then the slope steepened: since December 2025, scores have doubled every 3.0 months. Claude Opus 5.5 now sits at 2.30x the expert baseline, ahead of Fable 5.1 and GPT-6 Astra, at roughly 1/30 of the experts' cost per attempt. Research taste is the skill that decides how quickly AI could start improving AI, and this curve is one reason Jack Clark now expects that in 2027 (wrote about that yesterday as well).

2026.10.14 11:55

P-Zero Research's new TasteVal benchmark measures research taste: how well a model designs experiments and reads their results on eight real AI-development tasks, scored by how much compute it needs compared with human experts (the dashed 1x line). Progress was slow from GPT-4 in 2023 through 2025, then the slope steepened: since December 2025, scores have doubled every 3.0 months. Claude Opus 5.5 now sits at 2.30x the expert baseline, ahead of Fable 5.1 and GPT-6 Astra, at roughly 1/30 of the experts' cost per attempt. Research taste is the skill that decides how quickly AI could start improving AI, and this curve is one reason Jack Clark now expects that in 2027 (wrote about that yesterday as well).