VulcanBench
@VulcanBench
Benchmarking models across effort levels and harnesses on eval suites with real coding tasks. Because you don't need High or Max effort as much as you think.
๊ฐ์
March 2020
73 ํ๋ก์ ์ค 2.8K ํฌ
๐ VulcanBench v0.10.0 is out
Here's what's new:
โข Every run now logs its tool calls, one schema across 11 harnesses
โข vulcanbench tool-calls backfills older runs
โข New Mirror harness for Reflection Beam, with full traces
Now live on Github:
๋ ๋ณด๊ธฐ
