# Practices for Embedding AI Agents in Software # Risk-based Human Approval 🎯 The Hook "Let the agent do everything" and "approve everything manually" are both wrong. The answer is dynamic risk-based routing: auto-execute, require approval, or forbid -- based on each operation's actual risk. 🔥 The Problem When agents can execute operations with real-world side effects, full automation risks irreversible damage, while blanket approval requirements kill throughput. Telling the LLM to "ask if it seems dangerous" fails because that judgment itself is probabilistic and unreliable. 💡 The Pattern Classify every operation by risk score (irreversibility x failure cost) into three tiers. Low-risk operations (reads, reversible writes) auto-execute. Medium-to-high risk (irreversible or costly) requires human approval. Extreme risk (irreversible + catastrophic) is forbidden outright. Classification uses a deterministic rule engine, never the LLM itself. Approval timeouts default to rejection (safe side). An optional Autonomy Ladder dynamically adjusts thresholds based on the agent's track record over time. ✅ When to Use Use when: - The agent performs writes, deletes, sends, or other side-effecting operations - Operations vary in irreversibility and failure cost, making a uniform policy inadequate - A human review workflow exists and latency budget allows approval wait time Don't use when: - All operations are read-only with no side effects - Failure cost is uniformly low and rollback is easy - Latency requirements are too tight for human involvement ⚠️ Pitfalls - Never let the LLM classify risk. Classification belongs in deterministic code or a policy engine - Approval-pending state must be persisted; otherwise process restarts lose queued operations - Design for conditional approval (parameter modification) to avoid costly reject-and-resubmit loops 🔧 Implementation Approach - Classify risk using a static table (tool name x action) or a policy engine -- never the LLM itself - Compute risk score as (1 - reversibility) x failure_cost and map to three tiers (auto / approval / forbidden) via thresholds - Persist approval-pending state in a durable store (queue + persistence) so process restarts don't lose queued operations - Default approval timeout behavior to rejection (safe side) rather than auto-approval - Externalize risk policies as declarative config (YAML) so rules can be added or changed without code deploys #AIAgents# #SoftwareArchitecture#