Generative reward models are a promising direction for scaling up human judgment. Quality judgments for legal work product depend on many preference drivers like style, tone, writing quality, and document formatting that are less well-represented in current RLVR configurations vs. substantive and objective criteria. Legal outcomes are also inherently subjective — lawyers write documents called opinions for a reason. Trial judges and appellate judges disagree on 10-15% of cases. For this reason, our benchmarking + evals have historically relied on human lawyer preference judgments (SxS, Likert) which are high signal but low volume. In this article, @ItsJulioPereyra lays out how we’re scaling preference judgments with GRMs and their implications for both eval and training.

Model-harness co-optimization is helping us solve end-to-end legal use cases like M&A diligence. Together with @baseten and @baselabs, we built an RLM harness for M&A Diligence and are sharing initial results from post-training on top of it. The RLM harness lets a root agent delegate document review to sub-agents and aggregates their findings. A few initial results to highlight that we ran over LAB Diligence tasks: 1. Harness design — Across seven models, moving from a standard tool loop to our RLM harness increased mean criteria pass rate from 23.3% to 62.4%. 2. Post-training — In a separate experiment, RL over Qwen3.5 in the RLM harness more than doubled rubric pass rate, from 29.9% to 63.0%, and increased document review coverage from 62% to 96%. This makes the much smaller Qwen 3.5 competitive with the closed frontier in the RLM harness. We’re now doing a scaled RL run with GLM-5.3 and are excited about the potential to bring more capable models closer to completing these tasks end to end. Thanks to @oneill_c @mudithj and the @baseten @baselabs teams for the collab here!

Underrated interpretability result from our work with @EngramLab on @harvey law firm knowledge that makes a difference for the UX of long-horizon agents. The reasoning traces you get from parametric memory + study notes are much more intuitive than the interleaved bash commands you get from coding agents. When you read them (esp as a non-technical user), you can actually understand what’s going on. For example: "I remember two antitrust matters from earlier work, one that was terminated (client matter 1001-00004) and one that cleared after an HSR Second Request (client matter 1003-00003)" Compare this to the following from a coding agent (1 of 19 such bash commands btw): { "command": "cd /tmp/cache/1003-00003 && echo \"=== engagement letter head ===\" && sed -n '1,25p' Engagement/engagement-letter-hpe-fund-iv.docx.txt | cut -c1-500 && echo && echo \"=== practice group / partner mentions ===\" && grep -rhoi \"[^.]*\\(practice group\\|M&A partner\\|antitrust partner\\|responsible partner\\|lead partner\\)[^.]*\\.\" . | sort -u | head -12 | cut -c1-300 && echo && echo \"=== HSR outcome ===\" && grep -rhoi \"[^.]*\\(waiting period expired\\|expiration of the \\(extended \\)\\?waiting period\\|early termination\\|closed the investigation\\|closing letter\\|consent decree\\|no further action\\|without taking\\)[^.]*\\.\" . | sort -u | head -15 | cut -c1-400" } Interpretability like this becoming increasingly important for enterprise agent deployments. s/o to @dan_biderman @realJessyLin & team for innovating on multiple dimensions here.

An update on @harvey's post-training effort. We’re introducing Harvey Tenet, which is a culmination of research investments over the last 6 months across benchmarking, post-training, and open-weight models. Our model is trained specifically for long horizon legal work, but we also see broad generalization across other agent and legal benchmarks. We’re also highlighting new capabilities / specializations that have been unlocked through our research: 1. M&A Diligence: Post-training in RLM harnesses to enable models to effectively coordinate high-scale, long-horizon tasks 2. Review Tables: Making models more effective and efficient at high-volume document review and structured data extraction 3. Firm Knowledge: Models that are trained to understand firm knowledge through memory and emergent, structured taxonomies, enabling higher-quality and more efficient search Proud of the Harvey research team for this accomplishment and to be building alongside the frontier ecosystem: @FireworksAI_HQ @Kimi_Moonshot @NVIDIAAI @EngramLab @appliedcompute @mercor @trajectorylabs @SnorkelAI

Great article by @sonyatweetybird, esp on the importance of high-quality evals/benchmarks. "Most evals start as a founder squinting at outputs and vibe-checking whether they feel right." ^we started here but have since invested an enormous amount of time, effort, and resources into building Legal Agent Bench and other internal eval sets to turn legal judgment into a hill-climable signal. It's created a powerful flywheel for us @harvey. h/t @ItsJulioPereyra's team of legal researchers who bring their much needed domain expertise to the equation and at @BrendanFoody @mercor for helping us scale it up.

From @sequoia's own your intelligence event. We're starting to see a new category of company emerge, the Full-Stack AI company, where innovation happens at both the product and intelligence layer. That's why @sonyatweetybird led her opening keynote with: "The hottest new labs in my opinion are actually the applied research companies like @harvey @FactoryAI @glean @OpenEvidence @semgrep @tryramp" Proud of what the @harvey research and product teams have have accomplished across benchmarking, post-training, inference-time routing, agent harness optimization, scaling inference, open source, and more -- lots more to come here from this group

Always a blast collaborating with @NVIDIAAI and @trajectorylabs. For the GA launch of Nemotron 3.5 Lightning, we teamed up and put the same post-training recipe we ran for Nemotron 3 Ultra to the test.  With post-training, Nemotron 3.5 Lightning reached 8.3% on LAB, which is Opus-level all pass. Impressive for a model of this size. LAB post-training results are up there on the big board for the launch. The size of these bars relative the other benchmarks are a sign that we have a lot more work to do!

I've said this a few times but it's hard to explain in words how complex legal and professional work is. This time we're taking a different approach to conveying it — building a synthetic law firm. We’re introducing Calderwood & Harkness, a fictional law firm with 266 client matters across 46 clients, totaling ~10,000 documents and over 100M tokens of total context. It's about the size of a mid-market law firm. Together with @EngramLab we: - Built a persistent agent environment that mirrors how law firms organize their knowledge across clients, matters, and practice areas - Created 250 tasks that measure an agent's ability to search and reason over institutional knowledge at scale - Ran initial baselines showing that frontier models fail to exhaustively search firm knowledge out of the box Agents today re-derive the firm's knowledge from scratch on every task, which doesn't scale. We think the better path is to let agents build an understanding of the corpus up front, through indexes, summaries, and memory, and then reuse that understanding on every subsequent task. We’re thrilled to be partnering with @dan_biderman @realJessyLin and the @EngramLab team to explore this, more to share soon!

Impressive benchmark scores from Opus 5, inching closer to Fable-level performance on @harvey's Legal Agent Bench. Maybe more impressively, during our early access testing we found that Opus 5 performed much better at low reasoning than prior Opus checkpoints, leading to 26% gains in token efficiency relative those prior models. More here:

@nvidia continues to be an amazing partner in bringing open models to legal & professional knowledge work. Post-training Nemotron 3 Ultra on @harvey's Legal Agent Bench reached frontier quality at 10x lower cost per run.

Come help us scale @harvey’s model training team. If you’re interested in bringing frontier agent research into the Harvey product and working with: - @baseten to scale up RL to 80M+ token virtual datarooms - @PrimeIntellect to create structured agent training environments from unstructured legal data - @FireworksAI_HQ to navigate the quality <> cost Pareto frontier with inference-time routing and advisor models - @LangChain & LangChain Labs to build efficient verifiers and close the observability <> training feedback loop - @appliedcompute to post-train open weight models and high-volume agents for end-to-end legal tasks - @EngramLab to create an entire synthetic law firm and firm knowledge memory systems for better / more efficient open-world search - @trajectorylabs & @NVIDIAAI to shape the frontier of continual learning and sovereign AI for high-stakes domains - @mercor & @SnorkelAI to build out Legal Agent Bench and other benchmarks across legal and other verticals and other projects like this, then this is the role for you. Apply here:

One of the underrated pieces of our post-training work with @baseten is exploring KV cache compaction. In our training runs over Legal Agent Bench, agents converged to similar policies, roughly: 1. Ingest client matter documents up front 2. Analyze + iterate on intermediate work product 3. Draft final work product 4. Revise For heavier matters (real diligence datarooms can easily be 10M+ tokens), step (1) becomes the bottleneck. Natural language compaction is the logical starting point for long context problems, and we’ve shown a few times that NL compaction improves performance, but at 10M+ tokens even NL summaries get unwieldy. NL compaction is also lossy – the model is forced to compress a rich internal state into prose. Over long-horizon trajectories, subtle degradations of compacted context can compound. In legal, if you lose a key clause, date, or defined term the impact on work product is catastrophic. KV cache compaction is an interesting alternative. STILL, a method from Baseten Research, compresses the KV cache directly. For each layer, learned latent queries cross-attend over the full cache and write a much smaller set of synthetic keys and values in a single forward pass. The compressed state stays in a representation the model can attend to directly, rather than being forced through a text bottleneck. This has two cool properties 1. Because compression lives in the model's latent space, superposition lets each slot store significantly more information than a discrete token can. That gives the agent much denser working context than a memo or summary. 2. The compactor can be trained in-domain, so it can learn what kinds of details legal tasks actually need to preserve. Having the right memory primitives is increasingly part of the post-training equation for legal work, and knowledge work more generally. Much more to come here and s/o @oneill_c, @mudithj, and team for the collaboration and innovative work here.