Register and share your invite link to earn from video plays and referrals.

Ben Holmes
@BHolmesDev
The Whiteboard Guy and Factory Foreman @warpdotdev
1K Following    48.9K Followers
Stop reviewing agent PRs without verification! If there isn't a screenshot or screen recording, it's not worth your time. 10x more important than optimizing your CI checks.
We should stop keeping all of our agent conversations local. They should be stored in a shared place. This lets you do some powerful things: - Schedule self-improvement agents to review runs and pull out failure modes worth fixing - Replay past conversations against different models for benchmarking I met a team of devs zipping up Claude and Codex sessions from their computers and shipping them to a shared S3 bucket. That's... a start 😄 But really, all agent sessions should happen on a single source (like a cloud runner) so that conversation traces are stored together by default. It's why Warp Factories uses cloud sandboxes and makes sessions available via API. We also added an inbox view to see which conversations need your attention, which is quite convenient:
Show more
We made a skill for assessing the risk of merging a PR: /merge-risk This addresses the code review bottleneck by helping devs trust the agent for lower risk changes. It grades risk across 5 categories: alignment, impact, verification, rollback cost, and error detection. - Alignment: does code follow the PR description and related tickets? - Impact: what is the blast radius, and does it touch sensitive code paths? - Verification: does the change include end-to-end verification? This means computer use, screen recordings, before/after screenshots... going beyond unit test coverage. - Rollback cost: once it's shipped, how trivial is it to revert? @mattpocockuk calls this a "two-way door" vs. a "one-way door." - Error detection: are errors reported, logged, or simply swallowed? Try it out: > npx skills add warpdotdev/common-skills --skill merge-risk
Show more
At @aiDotEngineer, I presented our skill called /skill-doctor. It's a skill to improve your agent skills. It aggregates your past agent sessions, grades their quality, and propose improvements to fix common failures.
Show more
We benchmarked some agent orchestration pipelines at @aiDotEngineer. Which would you pick?
I've been thinking about what a true, closed-loop software factory will look like. One where agents know what to fix, what costs to optimize, and the big one: what features to actually build. We're not there yet. But maybe we're getting close...
Show more
Starting to think about engineering work as "product engineering tasks" and "factory engineering tasks." Product engineering: improve the product Factory engineering: improve the agents that improve the product I heard @poteto call this "gardening," which I really like! I think of these categories as inner and outer loops, where the inner loop is your team of agents building everything, and the outer loop is your team of supervisors / gardeners watching over those agents and spotting room for improvement. We're exploring how to automate that second loop. Right now, we have one "outer loop" of agents on our software factory that focuses on agent skill improvement. One batch of agents scores the quality of agent conversations: - was the agent efficient, or did it waste tool calls trying to understand its environment? - did the agent write good code by our standards, or was there, say, excessive test coverage? Then on a schedule, another agent looks for low-scoring conversations and suggests skill improvements based on them. We've merged quite a few skill changes so far, and the scores help us gauge whether those skill changes are helping. I'm interested to try another "outer loop" agents focused on code architecture / abstraction and adding static analysis checks. These are even better guardrails for agents than skills to prevent bad code viruses from spreading. Curious if other people are at setting up supervisor / gardener agents like this.
Show more
Code review is no longer mandatory. It’s now risk mitigation. Ask agents to explain the architecture of their changes to you. Ask questions. Have them ask *you* questions. Line-for-line PR review will become progressively less needed
Show more
Here's the multi-agent system driving our software factory right now. Core pieces: - We say what to build (sometimes Linear issues, mostly Slack threads) - Triage agent researches and decides whether to implement or to ask questions (think /grill-me but smarter) - Implementation agent builds, maybe collaborating with us on a spec with a subagent first - Verification subagent tests the implementation agent's work e2e - Code review agent cycles with the implementation agent for a few rounds before human review - Human reviews and ships - Monitoring automation responds to alerts and files issues (our first step to closing the factory loop) There's room to simplify this. We're benchmarking whether we need separate agents for steps like triage vs. implementation, or if a single high-intelligence model can do most of it. The core loop remains the same though. Curious how this compares to factories others are deploying
Show more
Skill GitHub has been achieved internally
Model selection is a huge driver for AI spend; we just reduced our cost-per-PR from $80 to $30 by switching from Opus 5 to GPT 5.6. We made this decision not by guessing, but by benchmarking. I want to show you how you can do this too! Join me for a live session on building custom model benchmarks: Thursday, September 17th 2pm ET We'll build a custom benchmark that replays your past agent conversations against a set of different models to find the best cost/performance pick. I'll whiteboard the entire setup so you can do this yourself, and we'll walk through the batteries-included version with Warp Factories. RSVP: 📅
Show more
I'm hosting a workshop next Thursday, Sept 3rd on how to build self improvement loops for agents. The idea is simple, yet super powerful: build agents that review past conversations, score the quality, and suggest skill improvements to fix inefficiencies. You'll walk away with a workflow you can apply to any setup, and an understanding of how this fits into a wider approach for software development with Warp Factories. Hope to see you there! RSVP: 📅
Show more
Introducing /skill-doctor v1 Skill doctor is a skill that improves your skills: - Crawls your past CC, Codex, or Warp convos - Scores them for efficiency and code quality - Proposes diffs to your skills that you'd actually merge (STE-100 approved) 🔖
Show more
0
61
2.5K
185
Forward to community
Been seriously impressed adding self-improvement to our agent conversations so far. We've merged 5 self-improvement PRs in the past few days, including this one that caught token burn in our message passing system for orchestration
Show more
We run all the agents in our company out of this one config file. Calling it... terraform for agents. Would you use it?
Working on a skill that improves your skills: /skill-doctor - Crawls your past CC, Codex, or Warp convos - Scores them for efficiency and code quality - Proposes diffs to your skills that you'd actually merge (hardest part I'm still tuning...) Would you use it?
Show more
🚨 Unpopular take: Slack is a pretty nice place to work with agents. - You can see how engineers smarter than you write their prompts - You can tag in designers to look before you ship something ugly - It's much easier to review screenshots and recordings that the agent sends
Show more
More CLI coding agents needs to support `cd`-ing around different directories while you're in a session. It's so freeing
I can't believe I'm still using Grok 4.5 (medium) as my daily. It's such a good default
0
175
1.3K
137
Forward to community
Introducing multi-agent orchestration in Oz, with support for Claude Code, Codex and the Warp Agent. Use /orchestrate to delegate complex tasks across a team of agents, locally or in the cloud.
Show more