# Practices for Embedding AI Agents in Software # Tiered Memory 🎯 The Hook Stuffing everything into the context window is a ticking time bomb: you'll hit token limits and permanently embed hallucinations in long-term memory at the same time. 🔥 The Problem When agents handle multi-turn tasks with flat memory, two failures happen simultaneously. Conversation history, user attributes, intermediate results, and retrieved knowledge compete for the same finite window, pushing out older context and breaking continuity. Worse, if the LLM's speculative outputs get persisted directly, hallucinations take root in long-term memory and contaminate every future session. 💡 The Pattern Separate memory into three tiers: working memory (in-context, ephemeral per turn), short-term memory (session store with TTL), and long-term memory (vector DB or KVS, persistent). Working memory is freely read-write and vanishes on context reset. Short-term entries carry trust tags distinguishing user-stated facts from LLM inferences. Promotion to long-term requires repeated confirmation or user approval, preventing hallucination from becoming permanent. The higher the failure cost, the stricter the promotion threshold and the longer the short-term TTL. ✅ When to Use Use when: - Information must carry across multiple sessions (user profiles, past decisions, accumulated knowledge) - Intermediate results are likely to exceed 30% of the context window - Distinguishing confirmed facts from speculation matters, and wrong memory has high downstream impact Don't use when: - Tasks complete in a single shot with no cross-session continuity needed - All information fits comfortably in the context window - Only write control is needed without tier separation ⚠️ Pitfalls - The boundary between working and short-term memory blurs easily. Use explicit external store writes as the dividing line, not LLM internal state - As long-term memory grows, irrelevant entries leak into context and trigger hallucinations. Filter search results by trust score - In multi-agent setups, letting individual workers write directly to long-term memory breaks consistency. Centralize long-term writes through the supervisor 🔧 Implementation Approach - Separate memory into three explicit layers: working memory (in-process dict), short-term memory (TTL-backed session store like Redis), and long-term memory (vector DB), using external store writes as the boundary - Implement recall as a budget-constrained search across all three tiers, ranking results by relevance and trust score to control injection volume - Gate promotion from short-term to long-term with a trust score threshold check and approval state verification, preventing unvalidated information from becoming persistent - Design TTLs per memory type (minutes for real-time data, weeks for user preferences, indefinite for immutable attributes), shortening TTLs as failure cost increases #AIAgents# #SoftwareArchitecture#