Tomek Korbak
@tomekkorbak
Joined June 2017
644 Following 11K Followers
understanding generalization is key to understanding misalignment. automating that understanding could be a powerful way to accelerate alignment research. we take a stab at this by using AIs to predict the alignment effects of training just by looking at the training data
Show more
Can you tell whether an AI will become evil by looking at its training data?
We built an AI forecaster that reads training data and predicts whether a model will exhibit various forms of misalignment, such as deception or power-seeking, before training.
It performs well above random guessing and beats several much stronger baselines 🧵
Paper:
Show more
