# Practical and Useful Patterns with ADK 🎭 Fixed-prompt tests can't replicate the unpredictable behavior of real users. ADK's User Simulation feature enables persona-driven conversational testing — "impatient customer," "vague requester" — to automatically verify agent robustness. 📌 Title: User Simulation 🔗 URL: 🧩 Overview ADK's User Simulation dynamically evaluates conversational agents using AI-generated user prompts. Instead of fixed test cases, a simulated user adapts to agent responses, making it ideal for testing unpredictable conversation flows. A ConversationScenario consists of three elements: `starting_prompt` (initial message), `conversation_plan` (high-level goals the user must achieve), and `user_persona` (user traits and interaction style). 🛠 Usage Basic user simulator configuration: ```json { "user_simulator_config": { "model": "gemini-flash-latest", "model_configuration": { "thinking_config": { "include_thoughts": true, "thinking_budget": 10240 } }, "max_allowed_invocations": 20 } } ``` ADK provides three pre-built personas: | Behavior | EXPERT | NOVICE | EVALUATOR | |----------|--------|--------|-----------| | Advance | Detail-oriented | Goal-oriented | Detail-oriented | | Answer | Relevant questions | All questions | Relevant questions | | Correct inaccuracies | Yes | No | No | | Troubleshoot errors | Once | Never | Never | | Tone | Professional | Conversational | Conversational | Define custom personas directly in ConversationScenario JSON: ```json { "user_persona": { "id": "IMPATIENT_USER", "description": "User in a rush, easily frustrated", "behaviors": [ { "name": "Short responses", "description": "Provides brief, incomplete replies", "behavior_instructions": [ "Keep responses under 10 words", "Omit polite phrases" ], "violation_rubrics": [ "Response exceeds 10 words", "Response is overly polite" ] } ] } } ``` Auto-generate evaluation scenarios: ```bash adk eval_set generate_eval_cases \ --user_simulation_config_file= ``` 🏗 Practical Patterns **Robustness Testing with "Impatient Customer" Persona**: Use an IMPATIENT_USER persona that gives sub-10-word responses and skips pleasantries. This tests whether your agent can gather sufficient information through proper follow-up questions despite receiving minimal input. Especially valuable for customer support agents. **"Vague Requester" Persona**: Simulate users who give abstract, unspecific requirements. Test whether your agent asks clarifying questions to pin down needs, rather than making assumptions or going off-track with ambiguous input. **Leveraging Pre-Built Personas**: EXPERT validates technical accuracy, NOVICE verifies that explanations are beginner-friendly, and EVALUATOR provides neutral performance assessment. Each reveals different aspects of agent quality. **Auto-Generated Scenarios for Coverage**: Use `generate_eval_cases` to automatically create diverse conversation scenarios, discovering edge cases you wouldn't think of manually. Specify `generation_instruction` for scenario direction and `environment_context` for backend state to produce realistic scenarios. 💡 Use Cases 🎭 Verifying customer support agent adaptability across diverse user types 🧪 Discovering edge-case conversation paths beyond fixed tests 🤖 Domain-specific robustness testing with custom personas 📊 Comprehensive quality evaluation through mass scenario generation ⚠️ Caveats - Without proper `max_allowed_invocations` settings, simulations may run indefinitely. Adjust from the default of 20 based on your domain. - Auto-generating scenarios requires the Vertex Gen AI Evaluation Service API. - Vague `violation_rubrics` in custom personas can lead to inconsistent persona behavior. Define specific violation conditions. - Enabling thinking in the simulator model produces more natural dialogue but increases costs. ✨ User Simulation breaks through the limitations of fixed tests. Use personas to simulate "how real users would behave" and verify agent robustness from multiple angles! #ADK# #AIAgent#