# Decision Points for Embedding AI Agents in Enterprise Systems
# Pre-verification vs Post-verification
๐ฏ The Hook
Are you applying the same verification policy to every agent output? Irreversible operations demand pre-verification, but applying pre-verification to information-only responses collapses throughput. If you do not design this fork based on the reversibility and impact of side effects, you will sacrifice either safety or speed โ unnecessarily ๐
๐ Overview
Pre-verification is the right choice when the agent's operations have irreversible side effects: payment execution, contract signing, data deletion, production deployment. Mistakes directly translate to financial damage, legal liability, or brand harm. Salary changes in Workday, contract amount modifications in Salesforce, and bulk price adjustments in Shopify are textbook examples. Pre-verification is also mandatory when regulatory compliance requires pre-execution approval, or when incorrect outputs cannot be tolerated โ medical, financial, or legal advice. Post-verification fits cases where output is informational only with no side effects, or where side effects are reversible and easily undone. Information summaries in Slack, document draft generation in Notion, and ticket classification suggestions in Jira fall into this category ๐
๐ Decision Points
Follow this flow:
Does the operation have side effects? โ No (information only) means post-verification is sufficient
Are the side effects reversible? โ Reversible means post-verification + automatic rollback works
Are the side effects irreversible? โ Irreversible means pre-verification is mandatory
What is the blast radius? โ Limited to one user vs. affecting all customers determines pre-verification intensity
Do regulations mandate pre-approval? โ Financial transactions, HR changes, and medical domains may require it
Can confidence scores be leveraged? โ Route below-threshold outputs to pre-verification, above-threshold to post-verification โก
๐ก Key Details
The standard production configuration is a dual-track setup combining pre and post verification:
Risk-level routing: Classify operations by risk upfront. High-risk operations (irreversible, high-impact) route to the pre-verification track. Low-risk operations (informational, reversible) route to the post-verification track. Routing is automatic.
Gradual relaxation: Start with all operations on pre-verification. As trust accumulates for specific categories, progressively migrate them to post-verification. Measure quality continuously and adjust thresholds.
Confidence score integration: Attach confidence scores to agent outputs and use threshold-based routing to automatically assign verification tracks. Set initial thresholds conservatively and calibrate with real data.
In Zendesk customer support, answer draft generation belongs on the post-verification track (sampling review), while refund processing belongs on the pre-verification track (human approval required) ๐
โ๏ธ Trade-offs
Putting irreversible operations on post-verification is the most dangerous design mistake. Bulk-deleting Shopify products and then reviewing the output is a path to unrecoverable disaster ๐ฉ
Putting everything on pre-verification is equally problematic. Requiring human approval for search result display drops agent response times to minutes, and users simply stop using the system.
Having no actual post-verification mechanism is another overlooked failure mode. Operations are classified as post-verification, but no one actually reviews them, and quality degrades silently. Define sampling rates and quality metric thresholds explicitly.
Stale risk classifications are dangerous too. Newly added operations default to post-verification when their risk level is undefined. Make risk classification a mandatory step whenever a new tool or API is added โ ๏ธ
๐ ๏ธ Use Cases
Customer support (Zendesk): Customer-facing answer draft generation runs on post-verification (10% sampling review for quality monitoring). Refund and credit processing runs on pre-verification (approver level varies by amount). Product exchange arrangements run on pre-verification (inventory check โ approval โ execution as a deterministic workflow) ๐
HR systems (Workday): Employee information search and display runs on post-verification (no side effects). Salary changes and transfer procedures run on pre-verification (regulatory requirements mandate approval). Org chart changes run on pre-verification with dual approval (broad impact scope) ๐
E-commerce operations (Shopify): Product information search and analysis runs on post-verification. Minor inventory quantity adjustments run on pre-verification (reversible but immediately impactful). Bulk price changes and product deletion run on pre-verification with dry-run and dual approval (irreversible, high-impact) ๐ง
Practical tip: Make "update the risk classification whenever a new tool or API is added" a mandatory team process. Unclassified operations defaulting to post-verification is not an intentional design โ it is an oversight ๐ช
#
AIAgents# #
EnterpriseArchitecture#