# Decision Points in AI Agent Development 🎯 The Hook Are you asking your LLM to calculate taxes? One of the most common mistakes in AI agent development is having the LLM reason through tasks that code could solve instantly. Wasted tokens, inconsistent results, impossible debugging. But the inverse is equally dangerous: trying to handle natural language interpretation with if-else chains until you drown in edge cases. Drawing the right line between LLM and code is essential. 📋 Overview For each processing step in an AI agent, the design question is whether to have the LLM reason in natural language or delegate to conventional code and tools. LLMs are probabilistic -- they may return different outputs for the same input -- but excel at interpreting ambiguous inputs and handling unforeseen cases. Code is deterministic -- same input always yields same output -- and is easy to test, but cannot handle cases not anticipated in advance. Correctly leveraging this asymmetry determines the system's reliability and cost efficiency. 🔍 Decision Points The heuristic is simple: can you write it as if-else? 📌 Fully describable by rules, procedures, or algorithms → Delegate to code. Tax calculation, inventory aggregation, date comparison, schema validation, format conversion. Having the LLM do these wastes tokens and risks errors. LLMs are notoriously bad at arithmetic and precise string operations. 📌 Input is natural language requiring intent interpretation → Let the LLM reason. Structuring a user's vague request is the LLM's sweet spot. Interpreting "reschedule tomorrow's meeting" -- what "tomorrow" means and what "reschedule" entails -- cannot be handled by if-else. 📌 Classification categories change dynamically, judgment criteria are context-dependent → LLM. When you cannot enumerate all cases in advance, code leads to edge-case explosion. 📌 Generation, summarization, translation, or other creative tasks → LLM. These are simply not achievable with code. 📌 Calculation errors are unacceptable → Code is mandatory. Financial calculations, permission checks, inventory management. Probabilistic LLMs should not be trusted here. 💡 Key Details 🔹 LLMs excel at deciding "what to do"; code excels at executing "how to do it." This asymmetry is the foundational design principle for agent systems. The most common and effective architecture has the LLM handle planning, classification, and interpretation while code handles computation, validation, and execution. 🔹 The litmus test: if the input determines a unique output, it is code's job. Calculations, lookups, format conversions, validations, database operations -- all code. 🔹 Symptoms of over-delegating to LLM: same input returns different results, triggering "the answer was different last time" complaints. Token spend far exceeds projections, and analysis reveals most is spent on code-replaceable tasks. High test flake rates destabilize CI/CD. 🔹 Symptoms of over-delegating to code: every new request type requires adding classification rules or parsers, slowing development. If-else chains balloon and become unmaintainable. Unexpected inputs get blanket error responses, rejecting requests the system could have handled. ⚖️ Trade-offs LLM gains flexibility and generality at the cost of reproducibility, expense, and debuggability. Code gains determinism and efficiency at the cost of flexibility and requires code changes for new cases. 🏆 The default is: if it can be solved deterministically, delegate to code. LLMs are expensive, inconsistent, and hard to debug. Limit LLM usage to tasks that code genuinely cannot solve. This is the core philosophy of "probabilistic judgment to the LLM, deterministic control to code." 🛠️ Use Cases 🧮 Expense reimbursement bot → Hybrid. Information extraction from receipt images and expense category classification go to the LLM. Tax calculation, total computation, and policy compliance checks are code. 🔍 Customer support → Hybrid. Intent classification of user inquiries goes to the LLM. Order status lookup, refund processing, and inventory checks are code (API/DB calls). 📊 Data analysis assistant → Hybrid. Interpreting "compare last month's sales by region" goes to the LLM. Actual query generation, execution, and aggregation are code. Summarizing and interpreting results goes back to the LLM. ⚠️ Anti-pattern: asking the LLM "what is 123 times 456?" LLMs can get arithmetic wrong. One line of `123 * 456` gives the guaranteed correct answer every time. #AIAgents# #SoftwareArchitecture#