# Decision Points for Embedding AI Agents in Enterprise Systems # General-Purpose Large Model vs Small Fine-Tuned Model — Choosing the Inference Engine 🎯 The Hook For your agent's inference engine, do you go with a GPT-4-class general-purpose large model controlled via prompts, or a task-specific small fine-tuned (FT) model? "Bigger is smarter" is true, but "bigger is always the right answer" is not. The optimal choice shifts based on task diversity, call frequency, cost constraints, and rate of change. Most enterprises find the practical sweet spot in a staged migration: start with a large model, accumulate training data from logs, then replace high-frequency tasks with FT models one by one 🔑 📋 Overview General-purpose large model plus prompting shines when tasks are diverse and cannot be enumerated in advance, when business requirements change rapidly, and when you want to handle new tools or APIs by simply updating prompts. Small FT models are appropriate when the task is narrowly defined with stable input/output patterns, call frequency is high and cost reduction matters, or latency requirements are strict. These are not mutually exclusive — a router plus specialist model hybrid is the standard enterprise approach 📊 🔍 Decision Points Here is the decision flow: If task types number 10 or more with new ones added frequently, choose the general-purpose large model. The flexibility to handle new tasks with just a prompt change is a decisive advantage. If monthly request volume exceeds several hundred thousand, calculate the cost benefit of small FT. Compare on total cost of ownership — not just API call costs, but also training costs, GPU inference infrastructure, and training data management overhead. If the latency requirement is P95 under 500ms, prioritize small FT. Large model inference times may not meet this bar. If you cannot prepare and continuously update several thousand or more training examples, stay with the large model. FT on insufficient training data will underperform the general-purpose model. If business requirements change more frequently than monthly, the re-training cost often makes maintaining the large model the better choice ⚡ 💡 Key Details Three hybrid patterns to understand: Router plus specialist models — Use the large model to classify request intent, then route routine tasks to small FT models. Intent classification cost is low, and specialist task inference cost drops dramatically. A Slack Bot takes company-wide questions, routes ticket classification to a small FT, and handles complex investigations with the large model. FT for drafts, large model for finishing — Generate drafts quickly with a small FT model, then verify and refine with the large model only for quality-critical cases. A practical pattern that balances cost and quality. Staged migration — Start with everything on the large model, accumulate training data from call logs, and replace high-frequency tasks with small FT models one by one. This is the lowest-risk approach and the one most enterprises adopt 🔄 ⚖️ Trade-offs Premature FT investment is the most common failure. Building an FT model before sufficient logs accumulate produces accuracy below the general-purpose model. Wait for at least three months of logs before considering FT. Reaching for FT during the PoC phase is premature 😩 FT model staleness is equally serious. Business rules and product lineups change, but training data is not updated, causing the model to answer with outdated knowledge. Automating the re-training pipeline is a mandatory investment for operating FT models. Cost comparison errors are frequent. Comparing only API call costs while overlooking FT training costs, GPU inference infrastructure costs, and training data management costs leads to flawed decisions. Compare on TCO (total cost of ownership) or the analysis is meaningless. Missing quality evaluation is another blind spot. Without regularly evaluating FT model accuracy, model drift goes undetected. Continuous A/B testing against the general-purpose model is the foundation of quality assurance ⚠️ 🛠️ Use Cases Company-wide help desk agent (Slack Bot): Handles broad queries across IT, admin, HR, and accounting. The large model classifies intent, then routes ticket categorization and prioritization (tens of thousands per month) to a small FT. Complex investigations and novel question types are handled by the large model 📚 Ticket classification agent (Zendesk integration): Automatically classifies category, priority, and assigned team for tens of thousands of support tickets daily. High call frequency and stable I/O patterns make small FT the optimal choice. Training data is updated monthly, and accuracy is monitored via A/B testing against the large model 🏢 Product review analysis agent (Shopify integration): Sentiment analysis and tagging of product reviews are batch-processed by a small FT. Customer insight summaries derived from reviews are generated by the large model. Routine classification goes to FT, creative analysis goes to the large model — a clear division of responsibility 🔧 Practical tip: "Start with the large model and accumulate logs" is the golden rule. Identify high-frequency tasks from three or more months of logs, build training data, validate FT effectiveness via A/B testing against the large model, then migrate. Make the FT transition a data-driven decision 💪 #AIAgents# #EnterpriseArchitecture#

# Decision Points in AI Agent Development 🎯 The Hook When something goes wrong, do you stop immediately or keep going at reduced quality? The uniquely difficult thing about AI agents is that outputs routinely land in a gray zone: "partially correct, partially wrong." Eight out of ten items are right, but two are inaccurate. Is returning that partial result helpful or dangerous? This is a different kind of judgment from traditional software's binary pass/fail. 📋 Overview This is a design decision about how AI agents behave when encountering errors or anomalies. Fail-fast means immediately halting and returning an explicit error. Graceful degradation means reducing quality or functionality while still returning some result. At any individual error point, this is a binary choice -- stop or continue -- with no middle ground. However, different error types can be assigned different policies. 🔍 Decision Points The key axes are the damage from acting on partial results and the latency budget. 📌 Partially incorrect results flowing downstream causes serious harm → Fail-fast. Financial transactions, medical decisions, legal document generation. The damage from partial errors being treated as correct far exceeds the cost of halting and retrying. 📌 Partial results are hard to distinguish from complete results → Fail-fast. If degraded output risks being mistaken for complete output, an explicit error is safer. False confidence is more dangerous than fail-fast. 📌 Partial results are better than nothing → Graceful degradation. When summarizing 10 documents and 2 fail to fetch, returning 8 summaries is more useful than just an error message. 📌 Quality degradation can be explicitly communicated → Degradation works well. When you can clearly state "1 of 3 sources could not be retrieved," partial results are genuinely useful. 📌 Accountability is required → Lean toward fail-fast. When you must explain "why this incomplete result was returned," explicit errors are easier to manage. 💡 Key Details 🔹 The essential value of fail-fast is preventing problem concealment. When a problem is detected, an alert fires immediately, enabling rapid root-cause investigation and resolution. With graceful degradation, problems can be masked and by the time they surface, the blast radius may have expanded significantly. 🔹 The essential value of graceful degradation is maintaining availability. In many scenarios, partial functionality is more useful to users than complete shutdown. Additionally, when fail-fast forces re-execution of the entire process, tokens consumed by the successful portions are wasted -- degradation can be more cost-efficient. 🔹 The biggest risk of graceful degradation: partial results flowing downstream indistinguishable from complete results. When downstream systems cannot differentiate "normal response" from "degraded response," batch processes run on incomplete data, or degraded states persist undetected for extended periods. ⚖️ Trade-offs Fail-fast gains safety and early error detection at the cost of reduced availability and retry expenses. When a long-running process is aborted at the final stage, all prior token spend is wasted. Graceful degradation gains availability and cost efficiency at the risk of incomplete results being treated as authoritative. 🏆 The proven approach is a degradation ladder with severity levels: Level 1 (minor): partial data missing → return the rest with explicit notation of what's missing Level 2 (moderate): primary model failure → switch to fallback model with explicit quality disclaimer Level 3 (critical): core data missing or integrity errors → fail-fast When in doubt, ask yourself: "If a user acts on this partial result, could irreversible damage occur?" If the answer is yes, fail-fast. 🛠️ Use Cases 💰 Financial transaction automation → Fail-fast. Partially incorrect trade instructions being executed is irrecoverable. An error message with retry is far safer. 📚 Document summarization service → Graceful degradation. When some document fetches fail, return summaries for what was retrieved with "summarized M of N documents" explicitly stated. More useful than an error message. 🤖 Conversational assistant → Graceful degradation. When one tool call fails, compose an answer from other sources with "some information could not be retrieved" appended. Far better UX than complete shutdown. 📊 Data pipeline → Fail-fast. Downstream batch processes running on incomplete data creates cascading impact. Stop, fix, and re-run is safer. 🏥 Medical decision support → Fail-fast. Judgment based on partial test results is dangerous. When required information is incomplete, return an explicit error and defer to human judgment. #AIAgents# #SoftwareArchitecture#

# Decision Points for Embedding AI Agents in Enterprise Systems # Optimistic vs Pessimistic Locking — Concurrency Control for Multi-Agent Systems 🎯 The Hook When multiple agents (or agents and humans) operate on the same resource simultaneously, how do you design the concurrency control? Do you take the optimistic stance — "collisions are rare, detect and retry" — or the pessimistic one — "lock before you touch"? The right answer depends on collision frequency and retry cost. And there is an agent-specific wrinkle: LLM inference time is non-deterministic, making lock hold time estimation harder than in traditional systems 🔑 📋 Overview Optimistic locking assumes collisions are rare. It skips locking during the operation, detects conflicts at write time via version numbers or ETags, and retries on collision. It excels at throughput and latency, but collapses when collisions become frequent. Pessimistic locking acquires a lock before the operation, blocking other agents until completion. It is suited for cases where consistency is paramount, but carries risks of deadlock and lock leakage. In enterprise environments, a hybrid that applies different strategies per resource type is the practical norm 📊 🔍 Decision Points Two axes drive this decision: collision frequency and retry cost. Estimate the concurrent access frequency for each target resource. If the collision rate is below roughly 5%, choose optimistic locking. Whether the operation is idempotent matters too. Idempotent operations make retries safe, favoring optimistic locking. If the operation has irreversible side effects, choose pessimistic locking (or combine with HITL approval). If retry re-computation cost exceeds acceptable bounds — long inference chains, external API side effects — adopt pessimistic locking. When adopting pessimistic locking, always design deadlock prevention measures (timeouts, lock ordering) ⚡ 💡 Key Details Hybrid patterns that mix strategies per resource are critical in practice: Strategy per resource type — Apply optimistic locking to read-heavy resources (knowledge articles, FAQs) and pessimistic locking to write-intensive resources (inventory, account balances). Matching the strategy to resource characteristics, rather than applying one uniformly, is the key. Escalation from optimistic to pessimistic — Run optimistic locking by default and automatically switch to pessimistic when a specific resource's collision rate exceeds a threshold (roughly 10%). Data-driven dynamic switching avoids excessive locking while maintaining safety. Short pessimistic lock with timeout — Set an upper bound on lock hold time (roughly 30 seconds) to prevent deadlock. After timeout, the lock auto-releases and the next agent picks up processing. An agent-specific consideration: LLM inference time is non-deterministic. Measure the P99 inference time and use that as the starting point for TTL configuration 🔄 ⚖️ Trade-offs Retry explosion under optimistic locking is a classic failure. Underestimating collision rates in batch processing where multiple agents update the same Salesforce records leads to cascading retries and system load spikes. Poor estimation translates directly to production incidents 😩 Deadlock under pessimistic locking is equally severe. Agent A locks resource X and requests Y, Agent B locks resource Y and requests X — this circular wait halts the entire system. Unified lock ordering and timeouts are necessary preventive measures. Lock granularity mistakes are common. Locking an entire table blocks processing of unrelated records. Record-level lock granularity should be the baseline. Lock leakage is especially dangerous in agent environments. When LLM inference terminates abnormally without releasing the lock, subsequent processing is blocked indefinitely. Use TTL-based locks (Redis SET ... EX) or lease-based mechanisms as countermeasures ⚠️ 🛠️ Use Cases Inventory management agent (Shopify integration): Multiple agents may decrement inventory simultaneously, so pessimistic locking prevents overselling. Lock TTL is set at 5 seconds accounting for API round-trip time. On timeout, the lock auto-releases and the task returns to the retry queue 📚 Opportunity management agent (Salesforce integration): Multiple agents rarely update the same opportunity's notes simultaneously, so optimistic locking (ETag) is used. When a collision is detected, the agent attempts a diff merge of the notes. Only unresolvable merge conflicts escalate to a human 🏢 Change management agent (ServiceNow integration): Change management records may be edited by a single agent over an extended period, making pessimistic locking appropriate. However, the lock hold time cap is 30 seconds — if LLM inference runs long, the lock is released and re-acquired to prevent blocking 🔧 Practical tip: Start by measuring actual collision rates on production data. Choose your locking strategy based on real data, not assumptions. And in agent environments, make "every lock must have a TTL" an ironclad rule. LLM inference can extend unpredictably, so TTL-free locks are simply not an option 💪 #AIAgents# #EnterpriseArchitecture#

# Decision Points in AI Agent Development 🎯 The Hook Are you asking your LLM to calculate taxes? One of the most common mistakes in AI agent development is having the LLM reason through tasks that code could solve instantly. Wasted tokens, inconsistent results, impossible debugging. But the inverse is equally dangerous: trying to handle natural language interpretation with if-else chains until you drown in edge cases. Drawing the right line between LLM and code is essential. 📋 Overview For each processing step in an AI agent, the design question is whether to have the LLM reason in natural language or delegate to conventional code and tools. LLMs are probabilistic -- they may return different outputs for the same input -- but excel at interpreting ambiguous inputs and handling unforeseen cases. Code is deterministic -- same input always yields same output -- and is easy to test, but cannot handle cases not anticipated in advance. Correctly leveraging this asymmetry determines the system's reliability and cost efficiency. 🔍 Decision Points The heuristic is simple: can you write it as if-else? 📌 Fully describable by rules, procedures, or algorithms → Delegate to code. Tax calculation, inventory aggregation, date comparison, schema validation, format conversion. Having the LLM do these wastes tokens and risks errors. LLMs are notoriously bad at arithmetic and precise string operations. 📌 Input is natural language requiring intent interpretation → Let the LLM reason. Structuring a user's vague request is the LLM's sweet spot. Interpreting "reschedule tomorrow's meeting" -- what "tomorrow" means and what "reschedule" entails -- cannot be handled by if-else. 📌 Classification categories change dynamically, judgment criteria are context-dependent → LLM. When you cannot enumerate all cases in advance, code leads to edge-case explosion. 📌 Generation, summarization, translation, or other creative tasks → LLM. These are simply not achievable with code. 📌 Calculation errors are unacceptable → Code is mandatory. Financial calculations, permission checks, inventory management. Probabilistic LLMs should not be trusted here. 💡 Key Details 🔹 LLMs excel at deciding "what to do"; code excels at executing "how to do it." This asymmetry is the foundational design principle for agent systems. The most common and effective architecture has the LLM handle planning, classification, and interpretation while code handles computation, validation, and execution. 🔹 The litmus test: if the input determines a unique output, it is code's job. Calculations, lookups, format conversions, validations, database operations -- all code. 🔹 Symptoms of over-delegating to LLM: same input returns different results, triggering "the answer was different last time" complaints. Token spend far exceeds projections, and analysis reveals most is spent on code-replaceable tasks. High test flake rates destabilize CI/CD. 🔹 Symptoms of over-delegating to code: every new request type requires adding classification rules or parsers, slowing development. If-else chains balloon and become unmaintainable. Unexpected inputs get blanket error responses, rejecting requests the system could have handled. ⚖️ Trade-offs LLM gains flexibility and generality at the cost of reproducibility, expense, and debuggability. Code gains determinism and efficiency at the cost of flexibility and requires code changes for new cases. 🏆 The default is: if it can be solved deterministically, delegate to code. LLMs are expensive, inconsistent, and hard to debug. Limit LLM usage to tasks that code genuinely cannot solve. This is the core philosophy of "probabilistic judgment to the LLM, deterministic control to code." 🛠️ Use Cases 🧮 Expense reimbursement bot → Hybrid. Information extraction from receipt images and expense category classification go to the LLM. Tax calculation, total computation, and policy compliance checks are code. 🔍 Customer support → Hybrid. Intent classification of user inquiries goes to the LLM. Order status lookup, refund processing, and inventory checks are code (API/DB calls). 📊 Data analysis assistant → Hybrid. Interpreting "compare last month's sales by region" goes to the LLM. Actual query generation, execution, and aggregation are code. Summarizing and interpreting results goes back to the LLM. ⚠️ Anti-pattern: asking the LLM "what is 123 times 456?" LLMs can get arithmetic wrong. One line of `123 * 456` gives the guaranteed correct answer every time. #AIAgents# #SoftwareArchitecture#