# Practices for Embedding AI Agents in Software # Tool Gateway / MCP Broker 🎯 The Hook Your AI agent calls multiple tools directly? That's a distributed security nightmare waiting to happen. A single gateway layer turns chaos into a controlled chokepoint. 🔥 The Problem When agents call external tools and MCP servers directly, authorization, rate limiting, and logging scatter across every integration. Prompt injection can sneak malicious arguments past individual tools, and audit trails become impossible to reconstruct when logs are spread across a dozen services. 💡 The Pattern Route all tool calls through a single gateway that enforces authorization, input sanitization, rate limiting, and audit logging in one place. Use dynamic scoping to expose only the tools relevant to the current task and user permissions, keeping the LLM's selection space narrow. Apply asymmetric policies: write operations get fine-grained per-operation authorization and HITL approval, while read operations use lighter category-level checks. Adding or removing tools becomes a configuration change, not a code deployment. ✅ When to Use Use when: - The agent calls multiple tools, at least one with side effects - User input or external data flows into tool arguments (low input trust) - You need an audit trail of who called what, with which arguments, and under whose authority Don't use when: - There's only one read-only tool and gateway overhead isn't justified - All tools are trusted internal services in an experimental environment where prototype speed matters more ⚠️ Pitfalls - The gateway itself becomes a single point of failure. Design health checks and a degraded mode (e.g., read-only fallback) - Never enforce authorization or sanitization via prompts. "Don't use this tool" instructions are trivially bypassed by injection - Session-level rate limits alone won't stop distributed attacks. Add a global rate limit layer on top 🔧 Implementation Approach - Define gateway policies declaratively (e.g., YAML), specifying type (read/write), authorization granularity, rate limits, sanitization rules, and log levels per tool - Dynamically scope tools exposed to the LLM based on task type, user permissions, and conversation phase, excluding irrelevant tools from the selection space - Design health checks and a degraded mode (read-only fallback) so the system survives gateway failures without total shutdown - Normalize schemas across MCP servers at the gateway layer, presenting a consistent interface to agents regardless of backend differences - Route high-risk code execution to sandboxed environments and use short-lived permission leases for long-running sessions #AIAgents# #SoftwareArchitecture#