# Practices for Embedding AI Agents in Software # Streaming with Progressive Commit 🎯 The Hook You want to stream tokens instantly for responsiveness, but committing side effects before validation is dangerous. What if you could separate "showing" from "doing"? 🔥 The Problem Agent responses have high latency variance, and making users wait for full generation degrades the experience. But committing tool side effects mid-stream means you need costly rollbacks when guardrails reject the output. You are stuck choosing between slow-but-safe and fast-but-risky. 💡 The Pattern Streaming with Progressive Commit pushes generated tokens and tool results to the client via SSE/WebSocket in real time, while holding side effects (API writes, DB updates) in a commit buffer until validation passes. Events flow from preview (pending) to committed or rejected, and the client UI explicitly renders intermediate states. The higher the failure cost, the deeper the buffer: high-risk workflows hold all side effects until the entire sequence is validated. ✅ When to Use Use when: - A user-facing UI exists and first-token-time directly impacts experience - The agent performs write side effects via tools, and reversal is costly - You need guardrail validation or dry-run checks before committing Don't use when: - Processing always finishes in a few seconds (streaming adds little value) - Clients cannot support SSE/WebSocket connections - The workflow is read-only with no side effects (commit buffer is unnecessary) ⚠️ Pitfalls - If the client does not distinguish preview from committed/rejected events, users see unconfirmed results as final. Always render a "pending" intermediate state in the UI - In long multi-step executions, the commit buffer can grow large. Checkpoint per step and release confirmed buffers to control memory - When an SSE connection drops, the commit buffer survives on the server. Decide upfront whether to restore on reconnect or discard after a timeout 🔧 Implementation Approach - Stream LLM tokens through a Stream Buffer directly to the client via SSE/WebSocket. Tool call results are accumulated in a separate Commit Buffer and sent as preview events - After generation completes, validate each buffered tool call against guardrails. Send committed events for passes and rejected events for failures - Execute tools initially as dry runs to produce previews, then commit only after validation passes, forming a two-phase execute-then-commit flow - Design SSE events with distinct types (token, preview, committed, rejected) so the client UI can render an explicit "pending confirmation" intermediate state - For high failure-cost workflows, keep the commit buffer deep and confirm all side effects only after the entire sequence validates. For low-risk cases, confirm per individual tool call #AIAgents# #SoftwareArchitecture#