An agent is three things: a harness, a model, and context. If you're serious about owning your intelligence, you probably want to own all three.
@LangChain founder
@hwchase17 joined us at our
@sequoia Own Your Intelligence to talk about the piece that often gets the least attention: the harness. He offers a clear heuristic for when to build your own. The more out of distribution you are from what the models were trained on, the more you'll want to customize.
And good technical content on how to actually measure performance with evals and langsmith.
00:00 Introduction
00:58 The three parts of an agent: harness, model, context
02:12 What a harness actually does
03:25 Customizing the core loop with middleware
04:41 Sandboxes, file systems, sub-agents, summarization
05:47 Cognitive architectures — and when you still need them
07:03 Build your own harness or use off the shelf?
08:24 In-distribution vs. out-of-distribution: the file-editing example
09:39 Why evals define what "good" means in an organization
11:04 Harbor: what an eval task actually looks like
12:11 Comparing harnesses and models on accuracy, latency, and cost
13:20 Why observability is underrated — it's usually the context
14:34 The data flywheel: traces → curation → experiments
15:42 Getting feedback through UX design and online evaluators
16:51 Demo: LangSmith Engine
19:23 Q&A: Running Engine on Engine, and "codex-ification"
20:44 Q&A: Will harnesses converge or diverge?