VulcanBench
@VulcanBench
Benchmarking models across effort levels and harnesses on eval suites with real coding tasks. Because you don't need High or Max effort as much as you think.
加入 March 2020
71 正在关注 2.7K 粉丝
Update on our Opus 5.5 benchmark.
Update on my Opus 5.5 benchmark with @VulcanBench
As many of you know, I don't get early access to these models, so I can only start benchmarking the day the model is released.
It looks like something in my eval suite is being refused by Claude Code's safety classifier, and is then falling back on Opus 4.8.
Looking into this more, reviewing the traces now. But likely going to be a bit longer for me to get the benchmarks out, stay tuned! 🖖
显示更多
