working on AGI alignment. prev: GPT-Neo, the Pile, LM evals, RL overoptimization, scaling SAEs to GPT-4, interp via circuit sparsity. EleutherAI cofounder.
the capacity to pause is very convergently useful.
even making an aligned AI that is superhuman at alignment research is not a sufficient win condition. it may successfully create a smarter AI, and then need a long time to align that AI, but race dynamics force yoloing sth half baked.