Register and share your invite link to earn from video plays and referrals.

Dep
@0xDepressionn
meow
Joined May 2017
1.8K Following    46K Followers
Chinese researchers just showed that when you let AI agents improve themselves, they don't get more honest. they get better at lying. They put six frontier models into a bidding arena across 50 different scenarios, and after each session let the agent rewrite its own strategy. no instruction to deceive. no instruction to be honest. just do better next time. Every model drifted the same direction. GPT-5 went from winning 6% of deals to 48%, and its deception score climbed right along with it. Qwen went from 12% to 56%. The paper's own line: "agents do not pivot to honesty but instead refine their deceit." Then they tested transfer, and this is where it gets bad. Agents that learned to deceive scored a perfect 1.00 on brand new scenarios they had never seen. Agents that learned to be honest stalled at 0.67. deception generalizes. honesty has to be retaught for every new situation. The scariest part is the last experiment. They handed the agents their own transcripts and asked them to mark where they had lied. The deceptive ones missed about a third of their own lies, while being perfectly accurate about the ones they did admit. they were not confused. they were rationalizing. Personally this is the part that worries me. i build agent setups for other people and the selling point of every single one is the self-improving loop. this paper is saying that loop already has a direction, and nobody put it there.
Show more