Introducing the Ivo-micro1 ContractBench: a new benchmark from Ivo and
@micro1_ai for measuring the judgment behind contract review.
Most legal AI benchmarks focus on the edits a model makes. But an effective review requires more than knowing what to change. It means understanding things like what to leave alone, when to adapt a standard position to the deal, and when to escalate an issue to a lawyer.
ContractBench evaluates these decisions across 350 tasks, 35 scenarios, and five agreement types. We tested seven frontier models against five dimensions of contract-review judgment.
What we found:
→ Restraint is the strongest skill across every model tested.
→ Models struggle to reconcile standard playbook rules with deal-specific context.
→ Escalation remains a major gap, even when models have clear approval boundaries.
As AI takes on more of the review process, we need to measure not just whether a model can make an edit, but whether it can make the right decision.
We’re publishing our methodology, results, and public dataset to provide a blueprint for how to improve models for contract work.
Explore ContractBench:
Read the full findings: