Eric Ho
@eric_ho
co-founder / ceo @GoodfireAI - AI interpretability research company
가입 September 2011
581 팔로잉 중 5.2K 팬
one underrated part of our reward hacking paper - we can detect when the model is contemplating a reward hack from its activations, before it even happens!
kind of like minority report
rollouts are stochastic though, so a flag means a higher chance of cheating, even if the model ultimately doesn't follow through
더 보기
We’re optimistic that every training run can be monitored and that the current rates of reward hacking may soon be a thing of the past.
Read the full post + paper:
