註冊並分享邀請連結,可獲得影片播放與邀請獎勵。

cv usk
@cv_usk
AI / Software Research Notes AI Agent, LLMOps, MLOps, Software Architecture 投稿は個人の意見です。
加入 May 2026
285 正在關注    451 粉絲
Practices for embedding AI agents into enterprise systems [Prompt Supply Chain Management] 💡 Has a "quiet prompt tweak" ever changed production behavior unexpectedly? This pattern treats prompts as code artifacts -- versioned, reviewed, signed, and eval-gated before deployment. 🔥 Problems it solves - Prompt changes cannot be tracked or reviewed, making it impossible to identify the cause of quality regressions - External inputs injected into prompt templates cause unintended behavior - No way to trace which prompt version produced a given result, destroying reproducibility 🏗️ Proposed pattern Manage prompts in Git with a dedicated registry. Require pull requests and code reviews for every change. Mandate eval gate passage on each update, automatically running quality checks against a golden dataset. Sign production-deployed prompts for tamper detection. Apply injection countermeasures -- delimiters, sanitization, and least-privilege scoping -- to all template variable insertion points. In essence, apply the same CI/CD quality process used for software to your prompts. ✅ Selection criteria - Good fit: Production agents with multiple contributors - Not a fit: Throwaway experiments or PoCs (governance overhead is not justified) ⚠️ Pitfalls - Without a culture that treats prompt changes as seriously as code changes, the process becomes hollow - Skipping signing and tamper detection leaves malicious or accidental changes undetectable - Failing to continuously update the golden dataset means new failure patterns slip through unnoticed 🛠️ Implementation Approach 1. Manage prompts, system instructions, and templates in a Git repository, requiring pull requests and code reviews for every change 2. Integrate an eval gate into the CI/CD pipeline to automatically run quality checks against a golden dataset on every prompt change 3. Sign production-deployed prompts with HMAC or similar mechanisms and verify integrity at deploy time for tamper detection 4. Implement injection countermeasures for external inputs in template variables -- delimiters, sanitization, and least-privilege scoping 5. Centralize version and deployment history in a prompt registry, enabling instant rollback when issues are detected in production #AIAgents# #EnterpriseArchitecture#
顯示更多