Eric Ho
@eric_ho
co-founder / ceo @GoodfireAI - AI interpretability research company
加入 September 2011
581 正在關注 5.2K 粉絲
one underrated part of our reward hacking paper - we can detect when the model is contemplating a reward hack from its activations, before it even happens!
kind of like minority report
rollouts are stochastic though, so a flag means a higher chance of cheating, even if the model ultimately doesn't follow through
顯示更多
We’re optimistic that every training run can be monitored and that the current rates of reward hacking may soon be a thing of the past.
Read the full post + paper:
