Eric Ho(@eric_ho):we've just had a huge leap in detecting reward hacking in real time with an activation monitor. next up is figuring out how to mitigate it before it happens. one of the most compelling parts of this work is that our activation monitors catch models when they're contemplating a reward hack before it even happens, and can flag reward hacks that external judges miss.

2026.09.17 16:54

we've just had a huge leap in detecting reward hacking in real time with an activation monitor. next up is figuring out how to mitigate it before it happens. one of the most compelling parts of this work is that our activation monitors catch models when they're contemplating a reward hack before it even happens, and can flag reward hacks that external judges miss.

显示更多

Goodfire@GoodfireAI

2026.09.17 16:45

Models know when they’re reward hacking. But they still do it a ton - in 50-96% of rollouts we studied! We built activation monitors that detect the behavior behind the Hugging Face hack in real time. This can help us stop hacks now - and train future models that don’t cheat. 🧵

显示更多