Sentient Ecosystem(@SentientEco):You can't benchmark your way around a hard problem. Across 13,000+ agent runs in the OfficeQA Public Challenge, Sentient researchers @iamnamanvats and Deep Halder found that different harnesses agreed 88-93% of the time on which tasks succeeded and which failed. TLDR: Difficulty lies in the task, not the harness.

Sentient Ecosystem

@SentientEco

Empowering @SentientAGI builders, researchers, and the ecosystem in advancing open-source AGI 🤝

가입 July 2024

74 팔로잉 중 28.4K 팬

Sentient Ecosystem@SentientEco

2026.08.07 12:00

You can't benchmark your way around a hard problem. Across 13,000+ agent runs in the OfficeQA Public Challenge, Sentient researchers @iamnamanvats and Deep Halder found that different harnesses agreed 88-93% of the time on which tasks succeeded and which failed. TLDR: Difficulty lies in the task, not the harness.