filipe
@filicroval
data eng | 1xAsus Ascent GX10 | benchmarking local models so you don't have to
Joined May 2023
214 Following 153.4K Followers
i don't blame models for not scoring on this benchmark, even I would probably fail miserably
GPT-5.6 Sol has been used to solve open problems in mathematics. So why was it struggling with ARC-AGI-3, a benchmark of 2D puzzle games?
We investigated. The harness was not letting it remember what it had learned.
We found that enabling two API settings tripled our scores with 6x fewer output tokens.
Show more
