# Practices for Embedding AI Agents in Software
# Prompt Registry / Prompt Artifact
🎯 The Hook
A one-word prompt change shipped inside a large commit caused a production incident. No version history, no rollback path, no way to tell which prompt produced which decision. Sound familiar?
🔥 The Problem
In LLM-based systems, a small prompt edit can dramatically alter model behavior. When prompts live as string literals scattered across application code, change tracking is impossible, regression testing doesn't exist, rollback requires a full code deploy, and auditors can't determine which prompt version drove a given decision. This is especially problematic in regulated domains where accountability demands a clear link between prompts and outcomes.
💡 The Pattern
Treat prompts as first-class versioned artifacts with the same rigor as application code: version control, peer review, automated regression testing via evaluation harnesses, and staged deployment. Roll out prompt changes through canary releases at 5-10% traffic, automatically rolling back on quality degradation. Record the prompt ID and version in every execution trace so post-incident audits can pinpoint exactly which prompt was active. Track prompt-model compatibility to identify which prompts need re-evaluation when the underlying model is updated.
✅ When to Use
Use when:
- Prompt change history and runtime version tracking are required (regulated industries, quality management)
- Prompts are referenced from multiple places and need centralized management
- You want A/B testing or gradual rollout capabilities for prompt changes
Don't use when:
- There are only one or two prompts with low change frequency in a personal project or PoC
- Output quality variation is acceptable in an exploratory context
⚠️ Pitfalls
- Template variables populated with user input need injection protection through escaping or sanitization
- Over-relying on an external registry API for prompt resolution introduces availability risk. Consider fetch-at-startup with local caching
- Store prompts in line-oriented formats like YAML or Markdown. A giant single-line JSON makes diff review nearly impossible
🔧 Implementation Approach
- Store prompts in line-oriented formats (YAML/Markdown) with template content, variable definitions, model compatibility constraints, and evaluation baselines in a single file
- Resolve prompts at runtime by ID from the registry, incorporating probabilistic canary routing to serve canary versions to a subset of traffic
- Record prompt ID and version in every execution trace, enabling post-incident audits to pinpoint which prompt drove each decision
- Integrate evaluation harnesses into CI to automatically run regression tests on prompt changes, then roll out via canary releases with automatic rollback on quality degradation
#
AIAgents# #
SoftwareArchitecture#