Okay, my Beam benchmark is done, and well, the results aren't very good...but it's also important to remember that @VulcanBench is focused entirely on agentic coding tasks.
Beam might be good at other things.
I am excited for where open models are going, and am betting that companies like Reflection have a bright future.
But there is one very interesting signal from this benchmark. Once you put Beam on a chart with models like Opus, Sol, and Astra, you see just how close the current frontier really is.
But the signal here is clear, this is not a model I would use for coding.
Full benchmark details will be added to the VulcanBench site soon.
Live long and benchmark ๐
๋ ๋ณด๊ธฐ
