# Practical and Useful Patterns with ADK 🧪 How do you verify that a prompt change hasn't degraded your agent's quality? ADK's Evaluation framework provides two approaches — test file method and dataset method — to systematically validate agent performance. Integrate it into CI to automate regression testing on every prompt change. 📌 Title: Evaluation Overview 🔗 URL: 🧩 Overview ADK provides two primary evaluation methods. The **test file method** uses `.test.json` files containing single sessions (with multiple turns) for unit-test-like fast iteration. The **evalset method** supports multiple lengthy sessions across many turns, suitable for integration testing. Some criteria (such as response quality, safety, and multi-turn quality) require the Vertex Gen AI Evaluation Service API. Built-in criteria include `tool_trajectory_avg_score`, `response_match_score`, `hallucinations_v1`, `safety_v1`, and more. 🛠 Usage Test files describe user queries, expected tool use trajectories, and expected responses. Three execution methods are available: **Web UI** for interactive evaluation: ```bash adk web path/to/agents_dir ``` **Pytest integration**: Call `await AgentEvaluator.evaluate()` with `agent_module` set to the target agent module name (e.g., `"home_automation_agent"`) and `eval_dataset_file_path_or_dir` pointing to the test file path (e.g., `"tests/integration/fixture/test.test.json"`). **CLI execution**: ```bash adk eval --config_file_path= ``` Configure evaluation thresholds in `test_config.json`: ```json { "criteria": { "tool_trajectory_avg_score": 1.0, "response_match_score": 0.8 } } ``` By default, tool trajectory requires 100% match while response matching accepts 80% similarity. 🏗 Practical Patterns **Regression Testing for Agent Updates**: Run existing test files after every prompt or tool definition change. `tool_trajectory_avg_score` quantitatively verifies tool selection accuracy, while `response_match_score` checks response quality. Early detection of quality degradation lets you iterate on prompts with confidence. **Fast Iteration with Test File Method**: During development, the `.test.json` approach works like unit testing. Define expected outcomes for single sessions, run them via pytest, and get quality feedback in seconds. Gradually expand test coverage by adding test files for each edge case. **Batch Evaluation with Dataset Method**: For complex, production-like scenarios, use the evalset method to evaluate many sessions at once. Multi-turn conversation flows and diverse user patterns can be comprehensively tested as integration tests. **CI Pipeline Integration**: Embed `adk eval` in your CI pipeline to automatically run quality checks on every pull request that modifies prompts. Fail the CI when scores drop below thresholds, preventing quality-degraded prompts from being merged. 💡 Use Cases 🔄 Automated regression testing on prompt changes ⚡ Fast development iteration with the test file method 📊 Comprehensive batch evaluation with the dataset method 🔗 Quality gate integration in CI/CD pipelines ⚠️ Caveats - Some criteria (such as response quality, safety, and multi-turn quality) require the Vertex Gen AI Evaluation Service API, which is a paid Google service. - Setting thresholds too high increases false negatives; too low risks missing quality degradation. Adjust incrementally from defaults. - LLM-based criteria (e.g., `hallucinations_v1`) can vary between judge model runs. Use `num_samples` for multiple sampling to improve reliability. ✨ ADK's Evaluation framework transforms agent quality management from intuition to quantitative benchmarks. Start with the test file method and progress to CI integration! #ADK# #AIAgent#