Register and share your invite link to earn from video plays and referrals.

艾略特
@elliotchen100
Raven: the deepthinking self-improving agent harness
Joined December 2023
1K Following    26.3K Followers
又来一篇论文。 这次我也有幸列在作者名单里。贡献边界得说清楚:真正把研究、工程和实验做出来的,都是算法同事。 我做的很少,更多是从产品和 Agent 生态角度参与了一些讨论,提供了一点问题定义和应用反馈,甚是荣幸。🫡 这篇论文研究的问题很实际: 网络上已经有几十万个 SKILL.md。把更多 skills 塞给 Agent,它就会自动变强吗? 团队从约 82.1 万个公开技能文件中,经过结构筛选、去重,以及质量、安全和许可检查,整理出 96,401 个技能。然后通过 retrieval、rerank 和 LLM selector,为每个任务选择 0–2 个真正相关的技能。 在三个 benchmark、两种 agent harness 和两种开源 backbone 下,SkillCorpus 都带来了正向提升,SkillsBench 上的汇总提升达到 +7.5 个百分点。 结论不是「skill 越多越好」,而是要在正确的任务里找到正确的 skill,再让 harness 真正把它执行出来。
Show more
Yet another paper. This time, we asked a practical question: Do public SKILL.md files actually make agents better? The open skill ecosystem has a quality problem. The hard part isn’t finding more skills. It’s knowing which ones are unique, useful, safe, permissively licensed, and relevant to the task. We crawled ~821K public skills and curated them into 96,401. Then we tested the full retrieval pipeline across 3 benchmarks, 2 harnesses, and 2 open backbones. It improved all three benchmarks, with the largest pooled gain at +7.5pp on SkillsBench. More skills are not automatically better. Curation, retrieval, and the harness determine whether skills become capability.
Show more