# Practices for Embedding AI Agents in Software # Prompt Registry / Prompt Artifact 🎯 The Hook A one-word prompt change shipped inside a large commit caused a production incident. No version history, no rollback path, no way to tell which prompt produced which decision. Sound familiar? 🔥 The Problem In LLM-based systems, a small prompt edit can dramatically alter model behavior. When prompts live as string literals scattered across application code, change tracking is impossible, regression testing doesn't exist, rollback requires a full code deploy, and auditors can't determine which prompt version drove a given decision. This is especially problematic in regulated domains where accountability demands a clear link between prompts and outcomes. 💡 The Pattern Treat prompts as first-class versioned artifacts with the same rigor as application code: version control, peer review, automated regression testing via evaluation harnesses, and staged deployment. Roll out prompt changes through canary releases at 5-10% traffic, automatically rolling back on quality degradation. Record the prompt ID and version in every execution trace so post-incident audits can pinpoint exactly which prompt was active. Track prompt-model compatibility to identify which prompts need re-evaluation when the underlying model is updated. ✅ When to Use Use when: - Prompt change history and runtime version tracking are required (regulated industries, quality management) - Prompts are referenced from multiple places and need centralized management - You want A/B testing or gradual rollout capabilities for prompt changes Don't use when: - There are only one or two prompts with low change frequency in a personal project or PoC - Output quality variation is acceptable in an exploratory context ⚠️ Pitfalls - Template variables populated with user input need injection protection through escaping or sanitization - Over-relying on an external registry API for prompt resolution introduces availability risk. Consider fetch-at-startup with local caching - Store prompts in line-oriented formats like YAML or Markdown. A giant single-line JSON makes diff review nearly impossible 🔧 Implementation Approach - Store prompts in line-oriented formats (YAML/Markdown) with template content, variable definitions, model compatibility constraints, and evaluation baselines in a single file - Resolve prompts at runtime by ID from the registry, incorporating probabilistic canary routing to serve canary versions to a subset of traffic - Record prompt ID and version in every execution trace, enabling post-incident audits to pinpoint which prompt drove each decision - Integrate evaluation harnesses into CI to automatically run regression tests on prompt changes, then roll out via canary releases with automatic rollback on quality degradation #AIAgents# #SoftwareArchitecture#