VulcanBench(@VulcanBench):Our benchmark of Reflection Beam on Frontier v4 is now out, and well, yeah, not great at coding. But I always like to point out that VulcanBench is a coding benchmark. Models that aren't designed to write code, probably won't do very well. Beam could be a great model at other things, so if you are looking at using it for non-coding related tasks, make sure to look at other benchmarks that have evals on the kind of things you're doing with the model.

2026.10.27 14:08

Our benchmark of Reflection Beam on Frontier v4 is now out, and well, yeah, not great at coding. But I always like to point out that VulcanBench is a coding benchmark. Models that aren't designed to write code, probably won't do very well. Beam could be a great model at other things, so if you are looking at using it for non-coding related tasks, make sure to look at other benchmarks that have evals on the kind of things you're doing with the model.

Morgan@morganlinton

2026.10.27 14:06

Okay, my Beam benchmark is done, and well, the results aren't very good...but it's also important to remember that @VulcanBench is focused entirely on agentic coding tasks. Beam might be good at other things. I am excited for where open models are going, and am betting that companies like Reflection have a bright future. But there is one very interesting signal from this benchmark. Once you put Beam on a chart with models like Opus, Sol, and Astra, you see just how close the current frontier really is. But the signal here is clear, this is not a model I would use for coding. Full benchmark details will be added to the VulcanBench site soon. Live long and benchmark 🖖