Eric Ho
@eric_ho
co-founder / ceo @GoodfireAI - AI interpretability research company
参加 September 2011
581 フォロー中 5.2K ファン
one underrated part of our reward hacking paper - we can detect when the model is contemplating a reward hack from its activations, before it even happens!
kind of like minority report
rollouts are stochastic though, so a flag means a higher chance of cheating, even if the model ultimately doesn't follow through
もっと見る
We’re optimistic that every training run can be monitored and that the current rates of reward hacking may soon be a thing of the past.
Read the full post + paper:
