Agents trained on population-scale data can do a lot of things. But ask a professional to stake their reputation on an AI-generated artifact? “Pretty good” isn’t good enough. We introduce TAHI: a Test-time Adaptive agent framework through Human-agent Interaction. Featuring: 📈 Test-time adaptation via context (memory, skills) and weight training ⚡ Efficient adaptation to individual expertise within tens of tasks ✔️ Creating comprehensive rubrics for “non-verifiable” tasks 🔍 Analysis of shared community guidelines vs. personalized tacit expertise

Great time chatting with @augmind_fm about adaptive AI agents! The meaning of "adaptive" in my research keeps shifting, but remains a missing recipe in agents. Back when agents were barely working, adaptive meant inducing reusable memory & skills from past experience, the idea behind two of my earlier "Agent Workflow Memory" ( and "Agent Skill Induction" ( It was surprising to see how much "memory", "skill", "workflow" resonates with the field in the years that followed. But the assumption behind agent adaptation has changed. Much of that previously missing knowledge is now baked into base model capability. Now the real gap between a generic agent and an ideal one, is adapting to a particular context, this could be a user, a company, etc. This is a harder problem than it sounds, because it has to happen efficiently: users often try a tool once or twice before giving up on it. It has to happen continuously: people's expertise and preferences drift, and an agent needs to detect that distributional shift and adapt to it in real time. It has to generalize beyond a particular user, ... These are underlying drives of my upcoming project, which I shared a few early thoughts on in the podcast 🤫 45:50 How do we build a real-time, continual agent adaptation framework? 48:20 How do agents upgrade simply through their interaction with humans? 49:02 How to comprehensively evaluate open-ended, 'non-verifiable' tasks? 51:10 Can agents integrate both shared community expertise and personalized expertise? What if agents could adapt while interacting with you, and do so fast enough within a few task sessions? A preview of what we're building: our agent translates various human interaction signals into context and weight updates, letting it effectively one-shot a solution tailored to the user's specific needs. More soon‼️