People of
@pidotdev! Many of you questioned what the heck does my extension actually bring to the table.
The other day I finally saw someone publish a bench I could compare to and run it with a model I can afford it to run on (thank you,
@melissapan). Two days later...
Turns out Pi with Pi-Codex-Conversion in notebook mode beats all the other harnesses. I used my settings: heavy sysprompt rewrite etc. No websearch, no skills.
The only caveat is it comes out slightly more expensive BUT: the env did not supply ripgrep somehow, and the first tool call was always rg with the outcome that it wasnt found. So I am assuming this woulda been slightly cheaper/efficient if it had all the tools. Sorry, I couldn't be arsed/didn't want to mess with the env too much.
Bosh. Any other questions?
Deets:
pi install npm:
@howaboua/pi-codex-conversion