登録して招待リンクを共有すると、動画再生報酬と紹介報酬を獲得できます。

Ivo
@heyivoai
The leader in AI contract review.
参加 March 2024
0 フォロー中    247 ファン
Introducing the Ivo-micro1 ContractBench: a new benchmark from Ivo and @micro1_ai for measuring the judgment behind contract review. Most legal AI benchmarks focus on the edits a model makes. But an effective review requires more than knowing what to change. It means understanding things like what to leave alone, when to adapt a standard position to the deal, and when to escalate an issue to a lawyer. ContractBench evaluates these decisions across 350 tasks, 35 scenarios, and five agreement types. We tested seven frontier models against five dimensions of contract-review judgment. What we found: → Restraint is the strongest skill across every model tested. → Models struggle to reconcile standard playbook rules with deal-specific context. → Escalation remains a major gap, even when models have clear approval boundaries. As AI takes on more of the review process, we need to measure not just whether a model can make an edit, but whether it can make the right decision. We’re publishing our methodology, results, and public dataset to provide a blueprint for how to improve models for contract work. Explore ContractBench: Read the full findings:
もっと見る