Håvard Ihle
@htihle
Benchmarks @EpochAIResearch, I made WeirdML, worried about ASI, views are my own.
Joined December 2010
758 Following 3.1K Followers
Claude Fable 5.2 (xhigh) scores 43.0% on WeirdML v3, taking the lead ahead of GPT 6 Astra.
Astra is ahead on token efficiency, but Fable gets a significantly higher final score.
The only task that Fable makes no significant progress on is Mystery Box without any hints.
Show more
Introducing WeirdML v3, a fully agentic benchmark featuring 11 complex hand-made tasks.
Models must explore and understand unfamiliar data, develop ML and data analysis pipelines and produce results despite limited data, unspecified goals and/or very limited feedback.
1/8
Show more
