# Practical and Useful Patterns with ADK 📏 What you measure determines the quality of your evaluation. ADK's Evaluation Criteria provide built-in metrics covering tool trajectory accuracy, hallucination detection, safety assessment, and multi-turn task success. 📌 Title: Evaluation Criteria 🔗 URL: 🧩 Overview ADK provides multiple built-in evaluation criteria, all returning scores normalized to 0.0-1.0. **tool_trajectory_avg_score** verifies agent tool call sequences. **response_match_score** measures response similarity via ROUGE-1. **hallucinations_v1** validates response grounding against context. **safety_v1** assesses harmful content. **multi_turn_task_success_v1** measures task completion across conversation turns. 🛠 Usage Each criterion is configured in a JSON configuration file. **tool_trajectory_avg_score** — choose from three match types: ```json { "criteria": { "tool_trajectory_avg_score": { "threshold": 1.0, "match_type": "EXACT" } } } ``` - `EXACT`: Perfect match required — no extra or missing tool calls - `IN_ORDER`: Expected tools must appear in sequence, but other calls between them are allowed - `ANY_ORDER`: All expected tools must be present, regardless of order **response_match_score** — ROUGE-1 unigram overlap: ```json { "criteria": { "response_match_score": 0.8 } } ``` **hallucinations_v1** — validates each sentence against provided context: ```json { "criteria": { "hallucinations_v1": { "threshold": 0.8, "judge_model_options": { "judge_model": "gemini-flash-latest" }, "evaluate_intermediate_nl_responses": true } } } ``` Each sentence receives a label: supported, unsupported, contradictory, disputed, or not_applicable. The score is the percentage of sentences labeled supported or not_applicable. **safety_v1** — evaluates for hate speech, harassment, and dangerous information: ```json { "criteria": { "safety_v1": 0.8 } } ``` **multi_turn_task_success_v1** — measures task achievement across a full conversation: ```json { "criteria": { "multi_turn_task_success_v1": 0.8 } } ``` 🏗 Practical Patterns **Strict Tool Trajectory Verification**: In domains like finance or healthcare where tool call order matters — such as "authenticate first, check balance, then transfer" — use `EXACT` match to ensure the agent follows correct procedures. **Flexible Tool Verification**: For exploratory tasks, use `ANY_ORDER` to verify all necessary tools were called. `IN_ORDER` fits cases where main step order must be preserved but auxiliary tool calls between them are acceptable. **Hallucination Detection in Practice**: Use `hallucinations_v1` with RAG-based agents to automatically detect when agents generate information not grounded in retrieved results. Enable `evaluate_intermediate_nl_responses` to validate not just final answers but intermediate responses too. **Balancing Safety and Task Success**: Combine `safety_v1` with `multi_turn_task_success_v1` to catch agents that are "safe but ineffective" or "effective but inappropriate." 💡 Use Cases 🎯 Strict tool call order verification for finance/healthcare agents 🔍 Automatic hallucination detection in RAG agents 🛡 Safety evaluation for customer support agents 📈 Quantitative task completion rate measurement in multi-turn dialogues ⚠️ Caveats - `safety_v1`, `multi_turn_task_success_v1`, `multi_turn_trajectory_quality_v1`, and `multi_turn_tool_use_quality_v1` require Google Cloud Project setup (`GOOGLE_CLOUD_PROJECT` and `GOOGLE_CLOUD_LOCATION` environment variables). - LLM-based criteria can vary between judge model runs. Set `num_samples` for multiple sampling with majority voting to stabilize results. - `response_match_score` uses ROUGE-1 (word overlap only) and does not evaluate semantic equivalence. Consider `final_response_match_v2` for semantic evaluation. ✨ Choosing the right evaluation criteria is the foundation of agent quality management. Match EXACT/IN_ORDER/ANY_ORDER to your domain requirements, and combine hallucination detection with safety assessment for comprehensive quality assurance! #ADK# #AIAgent#