DAIR.AI(@dair_ai):Giving an assistant extra tools seems harmless. Surprisingly, this paper shows it can make the assistant stop answering questions it knows. Across six models, the answer rate on questions that need no tool fell from 98.2% to 63.5% when a related tool was merely available. So the drop comes from the tool being present, even when it goes unused. The benchmark has 500 query pairs across 10 domains, each with a tool-unavailable control. A preceding tool call recovered some lost answers and caused new losses. A one-sentence system instruction that tells the model what the tool is for recovered up to 45.6 points. Paper:

2026.09.16 04:20

Giving an assistant extra tools seems harmless. Surprisingly, this paper shows it can make the assistant stop answering questions it knows. Across six models, the answer rate on questions that need no tool fell from 98.2% to 63.5% when a related tool was merely available. So the drop comes from the tool being present, even when it goes unused. The benchmark has 500 query pairs across 10 domains, each with a tool-unavailable control. A preceding tool call recovered some lost answers and caused new losses. A one-sentence system instruction that tells the model what the tool is for recovered up to 45.6 points. Paper:

显示更多