TL;DR A new benchmark called FDE-Gym recreates the ambiguous requests, hidden constraints, and off-target asks that real client-delivery engineers face. Even the strongest model solves only 15% of 95 tasks, and weaker models end up worse than doing nothing. Title: FDE-Gym: The Last Mile of AI Delivery URL: Points 🎯 95 tasks across 20 scenarios in 18 industries, each scored at 4 instruction levels from fully-specified rules to a possibly off-target original request 📉 The best model, Claude Opus 5.5, scores only 0.56, and solves just 14 of 95 tasks (15%) 🧩 The biggest bottleneck is discovering hidden constraints: models score 0.82 when rules are handed to them, but drop 0.28 once they have to find constraints themselves ⚠️ Weaker models like Claude Haiku 4.5 end up worse than a do-nothing baseline on 79 of 95 tasks 💸 Asking the client questions has a built-in "contact cost," and agents that over-ask lose 0.14-0.22 off their final score 📊 Models that hit 88-90% success on benchmarks like Terminal-Bench still fall to 0.19-0.56 on FDE-Gym The lack of correlation between cost or effort and the final score says a lot: working hard and actually improving the client's numbers are two very different things. #AIAgents# #Benchmarks#