Sentient Ecosystem(@SentientEco):You can't benchmark your way around a hard problem. Across 13,000+ agent runs in the OfficeQA Public Challenge, Sentient researchers @iamnamanvats and Deep Halder found that different harnesses agreed 88-93% of the time on which tasks succeeded and which failed. TLDR: Difficulty lies in the task, not the harness.

Sentient Ecosystem

@SentientEco

Empowering @SentientAGI builders, researchers, and the ecosystem in advancing open-source AGI 🤝

加入 July 2024

74 正在关注 28.4K 粉丝

Sentient Ecosystem@SentientEco

2026.08.07 12:00

You can't benchmark your way around a hard problem. Across 13,000+ agent runs in the OfficeQA Public Challenge, Sentient researchers @iamnamanvats and Deep Halder found that different harnesses agreed 88-93% of the time on which tasks succeeded and which failed. TLDR: Difficulty lies in the task, not the harness.

显示更多