登録して招待リンクを共有すると、動画再生報酬と紹介報酬を獲得できます。

cv usk
@cv_usk
AI / Software Research Notes AI Agent, LLMOps, MLOps, Software Architecture 投稿は個人の意見です。
参加 May 2026
286 フォロー中    470 ファン
You ask how to connect to the VPN, and your agent starts filing an IT ticket. In agent development, that happens more often than you'd think. An internal docs agent got a new create_ticket tool. All the ticket examples passed and the average score went up. But comparing example by example showed that "How do I connect to the VPN?" now filed a ticket instead of searching the docs. The trace showed why: the tool description said "VPN and laptop issues," which matched every VPN-related question. Narrowing it to requests that need IT to act made every example pass. Looking only at the average would have hidden the regression. That loop is Eval-Driven Development. Like test-driven development, you define what good looks like as a spec and an eval first, confirm it fails on the current agent, then compare experiments example by example after each change. A held-out set guards against overfitting, and production traces keep growing the evals. What Is Eval-Driven Development? How to Improve AI Agents #AIAgents# #LLMEvals#
もっと見る