Ian Tracey(@ian_dot_so):The world needs more vertical benchmarks that aren’t exposed in training data. Today we’re launching Accounting Bench focused on real world accounting tasks. What’s notable is even Fable and Astra can only fully solve ~20% of the tasks. There’s a lot of room to run.

2026.09.18 02:36

The world needs more vertical benchmarks that aren’t exposed in training data. Today we’re launching Accounting Bench focused on real world accounting tasks. What’s notable is even Fable and Astra can only fully solve ~20% of the tasks. There’s a lot of room to run.

显示更多

Ramp Labs@RampLabs

2026.09.17 16:05

Introducing Ramp Accounting Bench. We partnered with accounting professionals to create 137 tasks and grading rubrics grounded in everyday accounting workflows. Even with three attempts, the best model achieved only 21% accuracy. Reliable agentic accounting remains an open challenge.

显示更多