Vals AI(@ValsAI):EMB grades each task in two modes. In Template mode, the agent fills in a provided workbook skeleton and is scored on exact cell matches against a gold model. In Scratch mode, it builds the workbook from nothing, with an agent-as-judge scoring the workbook on numerical accuracy, formula wiring, and presentation. One mode tests whether an agent can follow the structure while the other tests whether it can create one.

Vals AI

@ValsAI

Public LLM Evaluation // @8vc @BloombergBeta @pearvc

参加 March 2024

269 フォロー中 16.2K ファン

Vals AI@ValsAI

2026.07.01 20:45

EMB grades each task in two modes. In Template mode, the agent fills in a provided workbook skeleton and is scored on exact cell matches against a gold model. In Scratch mode, it builds the workbook from nothing, with an agent-as-judge scoring the workbook on numerical accuracy, formula wiring, and presentation. One mode tests whether an agent can follow the structure while the other tests whether it can create one.