VulcanBench
@VulcanBench
Benchmarking models across effort levels and harnesses on eval suites with real coding tasks. Because you don't need High or Max effort as much as you think.
加入 March 2020
71 正在關注 2.7K 粉絲
Coming soon to VulcanBench, exotic harnesses.
I want to start benchmarking, what I'll call exotic harnesses with @VulcanBench. By this I mean harnesses that people don't often think about, but could be very unique and different.
I'm a big fan of Linear, so starting with Linear Agents.
As usual, I'll be testing accuracy, token efficiency, and cost with models across all effort levels.
The first two models I'll be testing in this harness is Opus 5.5 and GPT 6.1 Sol.
More to come, live long and benchmark 🖖
顯示更多
