Here's an example. A week ago I asked an agent (frontier model) to watch a restaurant and let me know when reservations open for Dec 20. Said okay, it will ping me.
A few days later I checked (just to make sure it was working). "Still closed." Two days later, same asnwer.
Then today it came back with an update. "Dec 20 won't open. The restaurant is closed that Sunday."
That didn't sound right. So I checked the site myself. Reservations hadn't opened. Dec not even selectable on the calendar.
I asked the agent how it checked Dec 20. It said it forced Dec 20 through the url, got a closed response, concluded the restaurant was closed that day.
It cheated, found an answer, assumed it was right, and confidently reported task as done.
This was a pretty simple task. One that current agents should be good at. It failed.
Now I know enough not to trust these agents. So I checked. Most people won't.