Tomek Korbak(@tomekkorbak):understanding generalization is key to understanding misalignment. automating that understanding could be a powerful way to accelerate alignment research. we take a stab at this by using AIs to predict the alignment effects of training just by looking at the training data

2026.09.25 19:34

understanding generalization is key to understanding misalignment. automating that understanding could be a powerful way to accelerate alignment research. we take a stab at this by using AIs to predict the alignment effects of training just by looking at the training data

John (Yueh-Han) Chen@jcyhc_ai

2026.09.25 18:57

Can you tell whether an AI will become evil by looking at its training data? We built an AI forecaster that reads training data and predicts whether a model will exhibit various forms of misalignment, such as deception or power-seeking, before training. It performs well above random guessing and beats several much stronger baselines 🧵 Paper: