登録して招待リンクを共有すると、動画再生報酬と紹介報酬を獲得できます。

Sumanth
@Sumanth_077
Simplifying LLMs, Machine Learning & AI Agents for you! • Building • Shipping Open Source AI Apps
参加 July 2021
871 フォロー中    76.7K ファン
Pytest for Google ADK agents! Testing AI agents is fundamentally different from testing regular code. There's no single output to assert against. Every run can produce a different response. And when something goes wrong, it's rarely obvious which step caused it. AI agents built using Google ADK run across multiple steps. A model call plans the action. A tool executes it. Another model call generates the final response. Any of these can quietly go wrong without an obvious error. DeepEval is an open-source LLM evaluation framework, for evaluating large-language model systems. DeepEval's Google ADK integration brings Pytest to this problem. One call to "instrument_google_adk()" and every agent run is automatically traced. Every model call, tool invocation, and agent step becomes a component span you can evaluate independently. Testing works at two levels. End-to-end evaluation scores the full agent run on task completion. Component-level evaluation attaches metrics to individual LLM calls or tool spans, so you know exactly which step failed when a test breaks. The pytest integration works the same way as any other DeepEval framework. Parametrize your test with Goldens, run the agent inside the test function, call "assert_test()" at the end. A failing metric fails the test, which fails the build. Run it with "deepeval test run". You can also run evals outside CI. Loop through Goldens in a script, run the agent, and score each resulting trace without touching pytest at all. Key capabilities: • Auto-instrumentation with one function call • Trace, agent, LLM, and tool spans all independently evaluable • Native pytest integration with assert_test() for CI/CD • Metrics: TaskCompletion, AnswerRelevancy, Faithfulness, G-Eval and more • Script-based eval outside of CI • Optional Confident AI dashboard for trace visualization 100% open source. I've shared the link in the replies!
もっと見る