VulcanBench
@VulcanBench
Benchmarking models across effort levels and harnesses on eval suites with real coding tasks. Because you don't need High or Max effort as much as you think.
Joined March 2020
71 Following 2.7K Followers
We have decided to stop work on our eval suite for System One Models.
Have a ton of respect for Diogo, and we're so busy benchmarking LLMs, if he doesn't want independent benchmarks to test models like Jev, we won't.
That being said, we love Jev, it's a great model, Diogo is a freakin genius and built a great team, highly recommend people try it for themselves and see how it might make its way into their workflow.
Onward, live long and benchmark đ
Show more
to re-iterate, I'm extremely anti-benchmarks (:
