Harness Engineering Practices P6. Multi-Layer Verification Pyramid (Match Verification Frequency to Loop Position) 🎯 Point Run a 10-minute E2E test every step and your agent's thinking stops every 10 minutes. Verification needs a "weight class" design. 📝 Overview Fast, cheap checks (type checking, lint) run every step. Medium checks (unit tests) run at milestones. Slow, expensive checks (integration, E2E) run at completion gates. Match verification cadence to loop position to minimize the latency tax. 🔍 Explanation Running all verification every step is inefficient; batching it all at completion is too risky. This practice applies test pyramid thinking to verification timing. Type checks and lint finish in seconds, so running them every step barely impacts latency. E2E tests take minutes and would severely hurt agent throughput if run frequently. By placing fast verification inside the loop and slow verification outside, you achieve both feedback speed and coverage. 🛠 How to Practice - Classify verification into 3 tiers: every step (type checks, lint — seconds), milestones (unit tests — tens of seconds), completion gate (integration, E2E — minutes) - Embed verification timing into the harness loop structure so each tier fires automatically at the right moment - Measure latency per tier and narrow test scope or parallelize if inner-loop verification is too slow - Periodically audit for coverage gaps between tiers and close them 💼 Use Cases - Pair programming: run only type checks instantly on each edit, run only affected tests - Issue-to-PR agents: verify with unit tests during implementation, run full CI before completion - CI auto-maintenance: triage with cheap checks, run full tests only for repair verification ⚠ Pitfalls Getting the layers wrong costs both speed and quality. Heavy checks run too often kill latency; only light checks let regressions escape. Overconfidence in lightweight checks ("type check passed, we're good") is also dangerous. Understand exactly what each layer covers and verify there are no gaps between layers. #HarnessEngineering# #TestingStrategy#