# Practices for Embedding AI Agents in Software # Read-Free / Write-Gated 🎯 The Hook Approving every single tool call is a recipe for approval fatigue, where the rubber-stamp on a dangerous write operation is just one click away. Separate reads from writes and focus human attention where it matters. 🔥 The Problem Agents mix side-effect-free reads with irreversible writes. Gating everything equally drowns humans in approval requests. Since reads dominate most workloads, approval fatigue sets in fast, and the critical write approvals get waved through without scrutiny. Remove all gates, though, and you risk irreversible damage from unchecked writes. 💡 The Pattern Split tool calls into "read" (search, fetch, reference) and "write" (create, update, delete, send). Let reads flow freely while gating writes with authorization, validation, approval, and audit. Classify R/W statically at tool registration time in code, never by LLM judgment. Graduate write gate strictness by reversibility: irreversible operations like email sends or payments require human approval, while reversible ones like draft saves pass through policy validation only. This dramatically reduces approval fatigue while maintaining safety for side effects. ✅ When to Use Use when: - Read and write operations are mixed, with reads making up the majority - Irreversible writes exist (email sends, payments, production DB changes) - You need to preserve human review bandwidth for high-risk operations Don't use when: - Reads themselves access sensitive data (PII lookups, confidential documents) and need authorization too - All operations are read-only with no writes at all - It's an experimental environment where all operations are reversible and low-cost ⚠️ Pitfalls - Never let the LLM classify read vs. write. Injection can make it label a write tool as "read," bypassing the gate entirely - Watch for "reads with side effects" like API call counters or view history tracking - Applying the same gate strictness to reversible and irreversible writes brings approval fatigue right back 🔧 Implementation Approach - Assign type (read/write) and gate mode (none/auto/human_approval) statically at tool registration, making it structurally impossible for the LLM to reclassify at runtime - Implement the write path as a pipeline of input validation, gate evaluation, execution, and full audit logging, while reads log only metadata - Graduate write gate strictness using a reversibility flag, combining irreversible operations with mandatory dry-run as a prerequisite - Enforce all gate logic in deterministic code at the gateway layer, with zero reliance on prompt-based access control #AIAgents# #SoftwareArchitecture#

# Practices for Embedding AI Agents in Software # Tool Gateway / MCP Broker 🎯 The Hook Your AI agent calls multiple tools directly? That's a distributed security nightmare waiting to happen. A single gateway layer turns chaos into a controlled chokepoint. 🔥 The Problem When agents call external tools and MCP servers directly, authorization, rate limiting, and logging scatter across every integration. Prompt injection can sneak malicious arguments past individual tools, and audit trails become impossible to reconstruct when logs are spread across a dozen services. 💡 The Pattern Route all tool calls through a single gateway that enforces authorization, input sanitization, rate limiting, and audit logging in one place. Use dynamic scoping to expose only the tools relevant to the current task and user permissions, keeping the LLM's selection space narrow. Apply asymmetric policies: write operations get fine-grained per-operation authorization and HITL approval, while read operations use lighter category-level checks. Adding or removing tools becomes a configuration change, not a code deployment. ✅ When to Use Use when: - The agent calls multiple tools, at least one with side effects - User input or external data flows into tool arguments (low input trust) - You need an audit trail of who called what, with which arguments, and under whose authority Don't use when: - There's only one read-only tool and gateway overhead isn't justified - All tools are trusted internal services in an experimental environment where prototype speed matters more ⚠️ Pitfalls - The gateway itself becomes a single point of failure. Design health checks and a degraded mode (e.g., read-only fallback) - Never enforce authorization or sanitization via prompts. "Don't use this tool" instructions are trivially bypassed by injection - Session-level rate limits alone won't stop distributed attacks. Add a global rate limit layer on top 🔧 Implementation Approach - Define gateway policies declaratively (e.g., YAML), specifying type (read/write), authorization granularity, rate limits, sanitization rules, and log levels per tool - Dynamically scope tools exposed to the LLM based on task type, user permissions, and conversation phase, excluding irrelevant tools from the selection space - Design health checks and a degraded mode (read-only fallback) so the system survives gateway failures without total shutdown - Normalize schemas across MCP servers at the gateway layer, presenting a consistent interface to agents regardless of backend differences - Route high-risk code execution to sandboxed environments and use short-lived permission leases for long-running sessions #AIAgents# #SoftwareArchitecture#

# Practices for Embedding AI Agents in Software # Model Router & Adaptive Effort 🎯 The Hook Running every request through your most powerful model is like dispatching a senior engineer to answer every support ticket. Route by difficulty, and you cut costs dramatically without sacrificing quality. 🔥 The Problem Processing all requests with the highest-performance model quickly blows through monthly budgets. But routing everything to a lightweight model causes quality collapse on complex reasoning and planning tasks. Large models also carry higher latency and heavier tail distributions, so using them for simple tasks unnecessarily degrades system-wide response times. Without model tiering, you're stuck in a cost-vs-quality tradeoff with no middle ground. 💡 The Pattern Dynamically select the model based on task difficulty, type, and risk. Start with a lightweight model; if confidence is low, escalate to a higher-tier model. Begin with a rule-based router (input length, task type, keywords) and upgrade to a classifier only if accuracy is insufficient. A 2-3 tier setup (small, medium, large) is a practical starting point. In production, 60-80% of requests typically resolve at the small or medium tier. ✅ When to Use Use when: - Task types are diverse, mixing routine extraction/classification with complex reasoning - A clear monthly cost ceiling exists - Traffic volume is high enough to recoup the routing mechanism's cost Don't use when: - All requests are similar difficulty -- a single model suffices - Quality can't drop even 1% on any request -- always use the top model plus caching - Request volume is low (hundreds per month) and routing development cost exceeds savings ⚠️ Pitfalls - Don't ignore the router's own cost. An LLM-based router can cost as much as one lightweight model call. Start with rule-based routing - Define confidence precisely. Use measurable indicators -- structured output parse error rate, refusal rate, internal log probabilities. "Seems uncertain" isn't operational - Prevent escalation infinite loops. Set a termination condition when even the top-tier model returns low confidence -- fall back to human escalation or error response 🔧 Implementation Approach - Start with a rule-based router (input length, task type, keywords) and only upgrade to a meta-classifier when classification accuracy proves insufficient - Abstract model tiers (small, medium, large) behind a common interface so providers can be swapped without changing routing logic - Run a confidence check on the lightweight model's response (parse error rate, refusal rate, log probabilities) and auto-escalate to a higher tier when confidence falls below the threshold - Cap escalation at the top tier -- if even the largest model returns low confidence, fall back to human escalation or an error response - Monitor routing ratios, cost, and latency on an ongoing basis to detect drift from model version changes and re-calibrate thresholds accordingly #AIAgents# #SoftwareArchitecture#

# Decision Points in AI Agent Development 🎯 **The Hook** Should your AI agent remember everything it hears? Memory write eagerness is one of the most delicate dials in agent design. Write too aggressively and you get memory pollution. Write too conservatively and your agent never learns. Getting this balance right is critical, and the stakes are higher than most teams realize. 📋 **Overview** Memory write eagerness controls how aggressively an agent persists information gathered during interactions into long-term memory. Think of it like a database INSERT: it is a quasi-irreversible operation. When an LLM's speculations or a user's ambiguous statements get written as facts, every future session references them as established truths. Errors become self-reinforcing. This is memory pollution, and it is one of the hardest problems to debug in production agent systems. 🔍 **Decision Points** This dial is primarily driven by two variables: 🔹 **Input Trust** — When end-user free-text is the primary source, the risk of injection and misinformation is high, so raise the write gate threshold. In admin-controlled input environments, you can afford to be more aggressive. 🔹 **Failure Cost** — In healthcare, legal, and financial domains, persisting incorrect facts leads to severe consequences. For an internal chatbot, a minor memory error can be corrected without much harm. Higher failure cost means stricter write suppression. 🔹 **Accountability** — When you need to explain "why was this stored in memory" after the fact, tracking provenance and confidence scores becomes essential. 💡 **Key Details** A practical three-tier framework for write decisions: ✅ **Auto-write** — Facts explicitly stated by the user ("My name is Tanaka," "I use Python") ⚠️ **Write after confirmation** — Information inferred from user behavior ("You seem to prefer Python" — confirm with the user before persisting) 🚫 **Never write** — LLM-generated speculation, unverified external sources, ephemeral context Attach confidence tags to memory entries and downrank low-confidence entries during retrieval. This limits pollution damage without completely blocking writes. Build deduplication into your write pipeline as well. Check new candidates against existing entries using cosine similarity (0.90-0.95 threshold), and overwrite same-entity same-attribute entries with the latest value. ⚖️ **Trade-offs** 📉 Too conservative — The agent never learns. Users repeat their preferences session after session, always getting default behavior. "I already told you this" becomes a recurring frustration. For use cases requiring long-term relationship building, this is a dealbreaker. 📈 Too aggressive — The biggest risk is hallucination persistence. "A-san probably lives in Tokyo" gets stored as "A-san lives in Tokyo" and treated as confirmed fact in all future sessions. Even more dangerous: prompt injection persistence. A single-session attack becomes a persistent injection when written to memory, affecting all future interactions. 🛠️ **Use Cases** 🏥 **Healthcare / Legal / Finance** — Extremely high failure cost. Minimize writes, record only explicitly confirmed facts, and always track provenance and confidence. 💬 **Customer Support** — Need to accumulate user preferences and history, but free-text input carries injection risk. Auto-persist only information confirmed through repeated interactions (2+ matches). Use a quarantine period for implicit preferences before promoting them. 🏢 **Internal Knowledge Bots** — Want to capture organizational tacit knowledge ("this API breaks if you pass this parameter"). Admin-controlled input allows more aggressive writing, but periodic "memory audits" where users review stored information maintain long-term quality. Never forget audit trails. Tracking when, what, and from which source each write occurred makes it possible to identify and fix the root cause when memory pollution is detected. #AIAgents# #SoftwareArchitecture#

# Practices for Embedding AI Agents in Software # Critic/Judge & Sampling-Aggregation 🎯 The Hook Asking an LLM to verify its own output is like asking the author to review their own paper. Separate generation from verification, and you structurally eliminate self-confirmation bias. 🔥 The Problem LLMs produce probabilistic outputs and hallucinate plausible-sounding falsehoods. When the same model and prompt handle both generation and verification, the biases carry over -- the model tends to affirm its own work. Ask the LLM that drafted a legal document "is this correct?" and it will likely say yes, even when it shouldn't. 💡 The Pattern Combine two complementary techniques. First, Critic/Judge: verify outputs using a separate model, separate prompt, or deterministic code to break self-confirmation bias. Second, Sampling-Aggregation: generate N candidates from the same input and select the best via scoring or majority vote, using probabilistic variance to your advantage. Together, this becomes "generate N candidates, have the Judge score each, pick the highest." Since cost scales linearly with N, apply this selectively to high-value, high-risk paths. ✅ When to Use Use when: - Incorrect output flowing downstream causes real harm (contracts, medical summaries, financial reports) - The economic value of a single request justifies the extra generation cost - Objective quality criteria exist (accuracy, consistency, schema compliance) Don't use when: - Minor errors are tolerable in casual conversation -- guardrails suffice - Latency budget is extremely tight (sub-second) - Evaluation criteria are subjective and hard to quantify (e.g., "creativity") ⚠️ Pitfalls - The Judge can share the Generator's blind spots. Same model family, same knowledge, same class of errors missed. Use a different model family or embed deterministic checks in the Judge - Increasing N adds cost linearly but with diminishing returns. The jump from N=1 to N=3 is significant; from N=3 to N=5, much less so. Start with N=3 - Specify explicit evaluation criteria in the Judge prompt. "Is this correct?" yields vague judgments. Provide concrete axes: factual accuracy, logical consistency, schema compliance 🔧 Implementation Approach - Separate the Generator and Judge into distinct models or distinct prompts, structurally preventing self-confirmation bias - Run N candidate generations in parallel to minimize the latency impact of sampling - Define the Judge's output as structured data (score plus reasoning), and implement the aggregation logic (best-score selection, majority vote) in deterministic code - Design a fallback path for when all candidates fall below the Judge's quality threshold (re-generation cap, human escalation) - Embed deterministic checks (schema validation, value-range checks, database lookups) as part of the Judge pipeline so verification does not rely solely on LLM judgment #AIAgents# #SoftwareArchitecture#

# Practices for Embedding AI Agents in Software # Deadline & Budget Cascade 🎯 The Hook Your agent spawns sub-tasks recursively, and suddenly the bill hits $50. "Billing accidents" happen when no node knows the global budget. There is a structural fix. 🔥 The Problem As an agent's call tree deepens, individual nodes have no idea how much resource they are allowed to consume. Expensive LLM calls stack up recursively, and plan-reflect loops retry endlessly even when improvement is unlikely. The root cause is that global budget and deadline constraints never reach local decision points. 💡 The Pattern Deadline & Budget Cascade sets a deadline (absolute timestamp) and budget (tokens, cost, step count) at the root of the call tree, then subtracts consumed resources before passing the remainder to each child task. Every leaf node knows exactly how much time and cost it has left, and can switch to degraded mode, return partial results, or halt before exhausting the budget. Deadlines are propagated as absolute timestamps, not relative seconds, to prevent drift accumulation across hops. ✅ When to Use Use when: - The agent recursively spawns sub-tasks or delegates to parallel workers - Per-request cost is unpredictable, risking billing accidents without caps - SLAs or time expectations exist for task completion Don't use when: - The call tree is flat (single level) and a simple timeout suffices - Batch jobs with no time pressure and predictable, fixed costs ⚠️ Pitfalls - Pass deadlines as absolute timestamps, not relative seconds. Relative values accumulate drift at each propagation hop (same principle as gRPC's grpc-timeout header) - Reserve a margin (10-20% of root budget) at the parent level. Aggregating and formatting child results requires its own time and cost - Decide exhaustion behavior upfront: partial result return, human escalation, or model degradation. Simply throwing a timeout exception destroys the user experience 🔧 Implementation Approach - Define a BudgetContext structure carrying deadline_at (absolute timestamp), max_cost_usd, max_steps, max_tokens, depth, and max_depth. Initialize it at the root when the user request arrives - On each child delegation, compute the child budget by multiplying the remainder by a fraction, subtracting a propagation margin (roughly 2s). Reserve 10-20% of the root budget at the parent for result aggregation - Every node checks remaining time, cost, and steps before proceeding, and switches to degraded mode, partial result return, or halt before the budget is exhausted - For parallel child tasks, note that cost is the sum across children while the deadline is governed by the slowest child. Distribute budget fractions by importance and estimated cost - Emit budget consumption ratio (consumed/limit) as a metric and trigger alerts when it crosses predefined thresholds #AIAgents# #SoftwareArchitecture#

# Decision Points in AI Agent Development 🎯 The Hook "Let's require approval for everything to be safe" is actually not safe at all. When 100 approval requests hit per day, approvers stop reading and just click OK. This is approval fatigue: the formal illusion of oversight with zero actual checking. It is more dangerous than having no approvals at all. HITL approval frequency is the dial that determines whether you get both safety and the benefits of automation. 📋 Overview HITL (Human-in-the-Loop) approval frequency controls how often an agent pauses to request human approval during processing. The spectrum ranges from approving every operation to approving only high-risk ones to relying on post-hoc sample audits. Approving everything throttles agent throughput down to human response speed, erasing the value of automation. Removing approvals entirely eliminates the last line of defense against hallucination-driven errors and unintended tool side effects. 🔍 Decision Points 📌 Failure cost: The primary driver. Higher-damage operations demand more approval; lower-damage ones can skip it. 📌 Reversibility: Whether an action can be undone changes the calculus entirely. Read operations and draft generation need no approval. Irreversible operations (production DB deletes, email sends, payments) require pre-approval. 📌 Approver cognitive load: Always keep in mind the paradox: higher approval frequency leads to lower approval quality. 💡 Key Details 🏗️ Use a risk-gate approach. Classify operations into three tiers: - Auto-execute: Read operations, reversible small writes. No approval needed. - Pre-approval: Irreversible operations, financial transfers, external notifications. Human confirms before execution. - Forbidden: Bulk production data deletion, etc. The agent is never granted execution rights. 🏗️ Batch approvals are highly effective. Reviewing 10 items in a list and approving them together is less taxing on the approver than 10 individual requests, and the ability to compare items actually improves review quality. 🏗️ Sample auditing adds efficiency. Randomly sample 5-10% of auto-executed operations for quality auditing. If anomalies surface, downgrade that category's autonomy level. 🏗️ Monitor approval fatigue quantitatively. If the average time from approval request to response drops below 2 seconds, approvers are likely not reading the content. ⚖️ Trade-offs 🔻 Too few approvals: Irreversible operations execute on LLM judgment alone. When hallucination-driven errors occur, it is too late. Lack of audit trails can also create compliance violations. 🔺 Too many approvals: Approval fatigue reduces effective checking to zero. Throughput becomes bottlenecked by human response speed: a 5-minute approval cycle turns a 30-second process into 30 minutes. Users grow frustrated with constant interruptions and abandon the agent entirely. The best practice is progressive trust building. Start with pre-approval, shift to sample auditing as track record accumulates, and promote to auto-execution once sufficient confidence is established. 🛠️ Use Cases 📧 Email sending agent: Draft generation needs no approval. Internal emails get sample auditing (10% post-check). Customer-facing emails require full pre-approval. Bulk sends (100+ emails) require multi-person approval. Templated order confirmations can be promoted to auto-execution. 🗄️ Database management agent: SELECT needs no approval. INSERT/UPDATE starts with pre-approval and transitions to batch approval with track record. DELETE always requires pre-approval. DDL operations require multi-person approval. Separating approval policies between production and development environments is essential. 🌙 24-hour batch agent: During off-hours when approvers are unavailable, queue approval-required operations for next business day. Default approval timeouts to the safe side (auto-reject). Auto-approve on timeout hollows out the entire approval process. #AIAgents# #SoftwareArchitecture#

# Practices for Embedding AI Agents in Software # Adaptive Timeout & Budget-Bounded Retry 🎯 The Hook Still retrying exactly 3 times? A single LLM retry can burn thousands of tokens. In the agent era, retry limits should be budgets, not counters. 🔥 The Problem Agent execution mixes operations with wildly different latency profiles: tool calls (seconds) vs. LLM inference (tens of seconds to minutes). A single fixed timeout either waits too long for tools or cuts off inference too early. Fixed-count retries ignore cost: three LLM retries can blow through your token budget, while three network retries may not be enough. And treating a 429 rate-limit the same as a schema violation wastes resources on retries that cannot succeed. 💡 The Pattern Adaptive Timeout & Budget-Bounded Retry sets timeouts per operation class (tool call, LLM inference, full session) and caps retries by remaining token budget rather than a fixed count. Errors are classified into three types: transient (429, 5xx) handled with exponential backoff, content-caused (schema violations) handled with self-correction, and context overflow handled with summarization or splitting. As budget consumption crosses thresholds, the system degrades gracefully: falling back to lighter models, returning partial results, and ultimately failing fast. ✅ When to Use Use when: - The agent combines operations with different latency characteristics - Retry cost is non-trivial (includes LLM inference) - Both transient network errors and content-caused errors can occur Don't use when: - A single LLM call with a fixed timeout is sufficient - Retries are prohibited (non-idempotent writes without idempotency keys) ⚠️ Pitfalls - Ignoring the Retry-After header on 429 responses and relying solely on your own backoff will trigger further throttling by the provider - Retrying non-idempotent writes without idempotency keys causes double execution. Either add keys or skip retries entirely for those operations - Stuffing raw error messages into self-correction prompts bloats the context window and triggers context-length errors. Summarize error feedback to roughly 200 tokens 🔧 Implementation Approach - Define timeouts per operation class: roughly 10-30s for tool calls, 60-120s total for LLM inference (with 5-15s inter-token timeout during streaming), and session-level deadlines for the full execution - Classify errors into three types and route accordingly: transient errors (429/5xx/timeout) use exponential backoff with jitter, content errors (schema violations etc.) trigger self-correction by appending an error summary to context, and context overflow triggers summarization or splitting - Cap retries by remaining token budget rather than a fixed count. Allow transient retries up to 90% of the budget and self-correction retries up to 70% - Deploy independent circuit breakers (Closed/Open/Half-Open) per provider, and on breaker open, descend a degradation ladder: lighter model, cached response, static fallback, then fail-fast - Implement a provider abstraction layer with a common interface so fallback routing is transparent to callers, and attach a degradation-level tag to response metadata #AIAgents# #SoftwareArchitecture#

# Practices for Embedding AI Agents in Software # Semantic Cache with No-Cache Zones 🎯 The Hook Paying for the same answer over and over? Semantic caching slashes costs, but cache the wrong thing and you'll serve stale stock prices or leak one user's data to another. 🔥 The Problem LLM calls are expensive, and when similar queries keep arriving, token costs pile up fast. Exact-match caching barely helps because of phrasing variations. But applying semantic caching indiscriminately creates serious risks: PII-dependent responses served to the wrong user, outdated real-time data, and safety-critical misjudgments replicated at scale. 💡 The Pattern Define "No-Cache Zones" by policy before anything else. Carve out PII-dependent, real-time, and safety-critical categories as forbidden zones. Only within the remaining safe zones does vector-embedding similarity matching apply. Similarity thresholds are tiered by risk level -- 0.92 for low-risk FAQs, 0.95 for medium, 0.97 for high-risk. Cache TTLs include 10-20% jitter to prevent thundering herd effects from mass expiration. ✅ When to Use Use when: - 20%+ of queries are semantically similar repeats - Per-request LLM cost is non-trivial - No-cache zones can be clearly defined by policy Don't use when: - Nearly all queries depend on user-specific context with no reuse potential - Failure cost is uniformly high, making any cached response unacceptable ⚠️ Pitfalls - Changing the embedding model invalidates the entire cache. Record model versions in metadata and plan migration strategy upfront - Weak pattern matching in No-Cache Zone classification lets forbidden queries slip through. Combine rules with an intent classifier in production - Cache poisoning risk: attackers can inject bad responses. Set quality score thresholds on cache writes 🔧 Implementation Approach - Start with rule-based No-Cache Zone detection (pattern matching + metadata flags), then add an intent classifier in production to handle phrasing variations - Set similarity thresholds per risk level, raising them as failure cost increases (starting points: 0.92 low-risk, 0.95 medium, 0.97 high-risk) - For medium-risk zones, add a lightweight revalidation step that verifies cache hits before serving - Record embedding model version in cache metadata and design a migration strategy (gradual re-embedding or flush) for model updates - Add jitter (10-20% random spread) to cache TTLs to prevent thundering herd effects from mass expiration #AIAgents# #SoftwareArchitecture#

# Practices for Embedding AI Agents in Software # Prompt Registry / Prompt Artifact 🎯 The Hook A one-word prompt change shipped inside a large commit caused a production incident. No version history, no rollback path, no way to tell which prompt produced which decision. Sound familiar? 🔥 The Problem In LLM-based systems, a small prompt edit can dramatically alter model behavior. When prompts live as string literals scattered across application code, change tracking is impossible, regression testing doesn't exist, rollback requires a full code deploy, and auditors can't determine which prompt version drove a given decision. This is especially problematic in regulated domains where accountability demands a clear link between prompts and outcomes. 💡 The Pattern Treat prompts as first-class versioned artifacts with the same rigor as application code: version control, peer review, automated regression testing via evaluation harnesses, and staged deployment. Roll out prompt changes through canary releases at 5-10% traffic, automatically rolling back on quality degradation. Record the prompt ID and version in every execution trace so post-incident audits can pinpoint exactly which prompt was active. Track prompt-model compatibility to identify which prompts need re-evaluation when the underlying model is updated. ✅ When to Use Use when: - Prompt change history and runtime version tracking are required (regulated industries, quality management) - Prompts are referenced from multiple places and need centralized management - You want A/B testing or gradual rollout capabilities for prompt changes Don't use when: - There are only one or two prompts with low change frequency in a personal project or PoC - Output quality variation is acceptable in an exploratory context ⚠️ Pitfalls - Template variables populated with user input need injection protection through escaping or sanitization - Over-relying on an external registry API for prompt resolution introduces availability risk. Consider fetch-at-startup with local caching - Store prompts in line-oriented formats like YAML or Markdown. A giant single-line JSON makes diff review nearly impossible 🔧 Implementation Approach - Store prompts in line-oriented formats (YAML/Markdown) with template content, variable definitions, model compatibility constraints, and evaluation baselines in a single file - Resolve prompts at runtime by ID from the registry, incorporating probabilistic canary routing to serve canary versions to a subset of traffic - Record prompt ID and version in every execution trace, enabling post-incident audits to pinpoint which prompt drove each decision - Integrate evaluation harnesses into CI to automatically run regression tests on prompt changes, then roll out via canary releases with automatic rollback on quality degradation #AIAgents# #SoftwareArchitecture#