注册并分享邀请链接,可获得视频播放与邀请奖励。

Mercor
@mercor
Organizing human intelligence to power the AI economy.
加入 April 2021
31 正在关注    30.9K 粉丝
Mercor partnered with @Reflection_ai on Beam's training and evaluation. Beam scores 48.8% Pass@1 on APEX-Agents, #1# among US open-weight models and closely trailing Chinese open models such as GLM-5.3-Flash (52.8%). On APEX-Agents, which measures long-horizon, multistep tasks in professional services, Beam scores a 15-point lead among US open-weight models. Beam: 48.8% Muse Spark Open: 33.4% Nemotron 3.5 Super VL: 29.5% Nemotron 3 Ultra: 22.7% It trails Qwen 3.8-Max by 14.5 points and lands within the confidence interval of GLM-5.3-Flash, trailing by 4.0 points. Qwen 3.8-Max: 63.3% MiMo-V2.6-Flash RL: 57.4% GLM-5.3-Flash: 52.8% Beam: 48.8% Kimi K3: 47.8% Qwen 3.8-27B: 47.5% GLM-5.2: 45.2% Compared with GLM-5.3-Flash on APEX-Agents domains, it leads on management consulting by 1.5 points. It trails GLM-5.3-Flash by 11.8 points on corporate law, and investment banking by 1.6 points. Congratulations to the Reflection team. We’re excited for future collaboration together advancing the US open source frontier. See full leaderboard:
显示更多
0
17
67
7
转发到社区