Congratulations to Team @harvey on launching a new agentic platform AND their own model. Faster, better, more natural AI power at your finger tips allowing your legal teams and law firms to do more faster and cheaper than ever before. @winstonweinberg @gabepereyra

as an engineer i never wanted to work at a healthcare company nine months ago i broke my own rule and sent this embarrassing email to OpenEvidence imo this is the most efficient ai company in the world $12b valuation with less than 35 engineers, so theres always something important to do i couldn't be happier

Great article by @sonyatweetybird, esp on the importance of high-quality evals/benchmarks. "Most evals start as a founder squinting at outputs and vibe-checking whether they feel right." ^we started here but have since invested an enormous amount of time, effort, and resources into building Legal Agent Bench and other internal eval sets to turn legal judgment into a hill-climable signal. It's created a powerful flywheel for us @harvey. h/t @ItsJulioPereyra's team of legal researchers who bring their much needed domain expertise to the equation and at @BrendanFoody @mercor for helping us scale it up.

One thing that often surprises people is that there are only 30(!) engineers at OpenEvidence and we're <100 people total. We look for a very specific profile: people that are relentless, autonomous, driven, and brilliant. Healthcare tech does not have to be slow, and AI is ushering in a new wave of capabilities. Join us!

Trajectory have generally impressed me with their tasteful execution on ambitious goals. on our Continual Learning track, @rronak_ gave a very thoughtful overview on how they're tackling the main data problems left in CL, including why GRPO isn't enough and they had to go on-policy.... and then subsequently fix all the issues that come up with it nice overview from one of the early leaders in this field! (see the rest of the track for more, this one was quite stacked)

A founder with deep technical clarity, strong conviction, and a vision to build something the world genuinely needs. Meet Arjun from Trajectory!- he's truly exceptional! @trajectorylabs @QuantumArjun

An agent is three things: a harness, a model, and context. If you're serious about owning your intelligence, you probably want to own all three. @LangChain founder @hwchase17 joined us at our @sequoia Own Your Intelligence to talk about the piece that often gets the least attention: the harness. He offers a clear heuristic for when to build your own. The more out of distribution you are from what the models were trained on, the more you'll want to customize. And good technical content on how to actually measure performance with evals and langsmith. 00:00 Introduction 00:58 The three parts of an agent: harness, model, context 02:12 What a harness actually does 03:25 Customizing the core loop with middleware 04:41 Sandboxes, file systems, sub-agents, summarization 05:47 Cognitive architectures — and when you still need them 07:03 Build your own harness or use off the shelf? 08:24 In-distribution vs. out-of-distribution: the file-editing example 09:39 Why evals define what "good" means in an organization 11:04 Harbor: what an eval task actually looks like 12:11 Comparing harnesses and models on accuracy, latency, and cost 13:20 Why observability is underrated — it's usually the context 14:34 The data flywheel: traces → curation → experiments 15:42 Getting feedback through UX design and online evaluators 16:51 Demo: LangSmith Engine 19:23 Q&A: Running Engine on Engine, and "codex-ification" 20:44 Q&A: Will harnesses converge or diverge?

Want world class research capabilities, but don’t have the resources of a big lab? At our recent Sovereign AI event, @gabepereyra shared @harvey ’s “moneyball” approach. Here’s the playbook: 00:00 Introduction 00:37 Building a research lab on a budget 02:28 Legal Agent Bench, contracting, and the diligence dataset 03:57 Domain experts guiding synthetic data generation 05:23 Why Harvey open sourced its datasets 06:55 Working with the neo labs – and why more than one 08:20 Post-training in-house: building "Associate 1" 09:44 The model serving matrix: 60 countries, fallbacks, SLAs 11:05 Deciding what stays in production 12:29 Simple open source switches and model routing 13:55 Moneyball: "If we win on this budget, we change the game" 14:53 Q&A: Training with sensitive data 17:16 Q&A: Competing for research talent 18:46 Q&A: Designing rubrics that actually challenge frontier models 20:19 Q&A: Where the pipeline breaks — data, research, or infra 22:59 Q&A: The tension in open sourcing a benchmark 25:02 Q&A: Biggest remaining open problems 27:10 Q&A: Competing with horizontal products

Intelligence and Experience are orthogonal vectors Terence Tao is perhaps the world’s smartest person, but drop him into an accounting firm or onto a construction site and on day one he’s not going to be very productive @trajectorylabs calls this The Experience Gap, and they have a way to close it @QuantumArjun explained how at our Sovereign AI event: 00:00 Introduction 00:12 Building the platform for continual learning 01:33 The experience gap: models have IQ but no tenure 02:52 Traceability → model spec → better models and harnesses 05:27 Four wishes for the agent ecosystem 06:34 Wish 1: Trace the whole tree — and capture the corrections 08:03 Wish 2: Evals from real traffic, graded in the real harness 09:26 Wish 3: Let the agents cook, and make tool responses informative 10:34 Wish 4: Get comfortable on open weights, experiment with routers 11:51 Why owning your intelligence shouldn't be consulted away 13:15 Demo: import a benchmark, train a model, deploy it 14:28 Q&A: What's the trainable object — weights, harness, or context? 16:08 Q&A: Continual learning without training on customer data 17:13 Q&A: Episodic memory and the hierarchy of feedback 19:37 Q&A: Where continual learning matters most

When should you start post-training your own models? @FireworksAI_HQ CEO @lqiao’s answer: after product-market fit. Not because it's hard... but because only after PMF is the data coming off your product surface worth training on. Lin joined us for our @sequoia "Own Your Intelligence" event to host a workshop on all things post-training; what works, what breaks, and how not to let the model outsmart you. Must listen!! 00:00 Introduction 00:37 What Fireworks sees across thousands of AI applications 02:47 Off-the-shelf APIs and the problem of keeping your taste 03:58 What "owning your intelligence" actually means 05:43 The progression: prompting → RAG → SFT → preferences → RL 07:20 Why this mirrors how humans learn 09:03 Matching the technique to the problem you actually have 10:46 Where teams get stuck: data quality and vibe evals 12:28 Reward hacking: the model that wrote zero lines of code 13:59 Training-to-serving alignment (and why quality drops) 15:55 Post-training in healthcare and security 17:31 From coding to every co-work domain 19:35 Incumbents, cost burden, and not scaling into bankruptcy 21:26 How much control do you want? 23:24 Q&A: What makes a good reward signal 25:00 Q&A: When to start thinking about post-training

A model is only as good as its data, and we’ve long since exhausted the internet. From here on out, model progress is gated by data production. @mercor’s @BrendanFoody joined us at our Sovereign AI event to talk about how RL environments get built, and why your data might be your real moat: 00:00 Introduction 00:47 A short history of the data market: crowdsourcing to agentic data 02:29 What an RL environment is: worlds, apps, tasks 03:57 Why only humans can measure the frontier 05:35 Building verifiers is the hard part 06:44 Walkthrough: a real legal RL environment 08:18 Leaderboards — and what open weights change 09:45 Post-training results on Apex Agents 11:17 Three ways companies buy data 12:49 Q&A: How do you price data? 14:17 Q&A: What "data quality" actually means 16:42 Q&A: The misunderstanding about synthetic data 18:17 Q&A: Why RL environments now — and what comes after 21:20 Q&A: Can you scale rubric generation with models? 23:00 Q&A: RL environments for cyber defense 25:33 Q&A: Build data in-house or partner?