micro1(@micro1_ai):The most advanced AI models still get nearly half of expert contract review judgments wrong. We partnered with @heyivoai to examine why, and where even the most advanced models fall short when reviewing and negotiating real contracts. Introducing the Ivo-micro1 ContractBench, a benchmark built by practicing attorneys to test how well AI handles the judgment calls that come with reviewing a contract, not just whether it can draft. No model scored above 59%. Models did well on the straightforward calls, like leaving acceptable language alone or applying a clear playbook rule, but they struggled most on the issues attorneys considered highest priority. Knowing what to push back on, what to leave alone, and when to escalate proved challenging for the models evaluated. Explore the full results and methodology below.

2026.10.20 17:18

The most advanced AI models still get nearly half of expert contract review judgments wrong. We partnered with @heyivoai to examine why, and where even the most advanced models fall short when reviewing and negotiating real contracts. Introducing the Ivo-micro1 ContractBench, a benchmark built by practicing attorneys to test how well AI handles the judgment calls that come with reviewing a contract, not just whether it can draft. No model scored above 59%. Models did well on the straightforward calls, like leaving acceptable language alone or applying a clear playbook rule, but they struggled most on the issues attorneys considered highest priority. Knowing what to push back on, what to leave alone, and when to escalate proved challenging for the models evaluated. Explore the full results and methodology below.

Ivo@heyivoai

2026.10.20 17:00

Introducing the Ivo-micro1 ContractBench: a new benchmark from Ivo and @micro1_ai for measuring the judgment behind contract review. Most legal AI benchmarks focus on the edits a model makes. But an effective review requires more than knowing what to change. It means understanding things like what to leave alone, when to adapt a standard position to the deal, and when to escalate an issue to a lawyer. ContractBench evaluates these decisions across 350 tasks, 35 scenarios, and five agreement types. We tested seven frontier models against five dimensions of contract-review judgment. What we found: → Restraint is the strongest skill across every model tested. → Models struggle to reconcile standard playbook rules with deal-specific context. → Escalation remains a major gap, even when models have clear approval boundaries. As AI takes on more of the review process, we need to measure not just whether a model can make an edit, but whether it can make the right decision. We’re publishing our methodology, results, and public dataset to provide a blueprint for how to improve models for contract work. Explore ContractBench: Read the full findings:

Forward to community