Florian Brand
@xeophon
evals @PrimeIntellect | open models @interconnectsai
๊ฐ์
July 2015
793 ํ๋ก์ ์ค 15.9K ํฌ
oh would you look at this, an open model being sota ๐
sinatras cooked so hard here ๐จโ๐ณ
Time to give agents a hard task, introducing pmpp-hard!
69 GPU kernel tasks, 11 models, 3.1k agent rollouts and 5.8B+ tokens later we have the results. Kimi K3 claims the first place with a 0.71 score. Without dealing with if your task is โfrontierโ or not, it just solves them
๋ ๋ณด๊ธฐ
