注册并分享邀请链接,可获得视频播放与邀请奖励。

cv usk
@cv_usk
AI / Software Research Notes AI Agent, LLMOps, MLOps, Software Architecture 投稿は個人の意見です。
加入 May 2026
282 正在关注    431 粉丝
# Useful but Little-Known Features of ADK 2.0 🌍 What if you could replay realistic user conversations to evaluate your agent — without needing actual users? ADK 2.0's user simulation feature dynamically generates conversations with virtual users of varying personas, enabling systematic evaluation of your agent's response quality. 📌 Title: User Simulation 🔗 URL: 🧩 Overview User simulation automatically generates dynamic conversational prompts for agent evaluation. You define a `ConversationScenario` with a `starting_prompt` (the opening message), a `conversation_plan` (how the conversation should progress), and a `user_persona` (the simulated user's behavior). Three built-in personas are provided: EXPERT (detail-oriented, corrects the agent's mistakes), NOVICE (passive, waits for prompts and guidance), and EVALUATOR (detail-oriented but does not correct errors). You can also create custom personas with `UserPersona`. 🛠 How to use it Use the CLI command `adk eval_set generate_eval_cases` to generate evaluation cases. For custom personas, use the `UserPersona` class. ```python from google.adk.evaluation.conversation_scenarios import ConversationScenario from google.adk.evaluation.simulation.user_simulator_personas import ( UserBehavior, UserPersona, ) # Define a custom persona impatient_user = UserPersona( id="IMPATIENT", description="Prefers short answers and shows frustration with lengthy explanations", behaviors=[ UserBehavior( name="Short responses", description="The user provides very short, sometimes incomplete responses", behavior_instructions=["Keep your responses under 10 words.", "Omit polite phrases."], violation_rubrics=["The user response is over 10 words."], ) ], ) # Define a scenario scenario = ConversationScenario( starting_prompt="Help me book a trip", conversation_plan="Confirm destination → dates → budget, then complete the booking", user_persona=impatient_user ) ``` 🏗 Building it into production ・Integrate evaluation scenarios into CI/CD pipelines to run regression tests automatically on agent updates ・Use all three built-in personas (EXPERT, NOVICE, EVALUATOR) to cover different user segments ・Design `conversation_plan` around real business workflows to match actual usage patterns ・Create custom personas that mirror your product's specific user behavior patterns 💡 Use cases 🧪 Automatically detect conversation quality regressions after prompt or model changes 👥 Verify that different response strategies work for beginners vs. power users 📊 Run batch evaluations across multiple scenarios to systematically identify agent weaknesses 🔄 Test how well the agent handles topic shifts mid-conversation or ambiguous questions ⚠️ Watch out Since simulations are LLM-generated, the same scenario may produce slightly different conversation flows on each run. For reproducible evaluations, run multiple iterations and assess results statistically. Also, each built-in persona behaves quite differently — make sure you select the one that matches your evaluation goals. ✨ User simulation lets you continuously improve your agent's conversational quality without involving real users. Make multi-persona evaluation a regular part of your development workflow. #ADK# #AIAgent#
显示更多