New paper: Adding clean data to your dataset is assumed to weaken poison data. We show the opposite can occur. Clean data can *strengthen* backdoor misalignment, if the poison has an identifying trigger. We call this homeopathic learning.

My favorite experiment here is subliminal transfer of a random smooth function. We trained an LLM to predict a random small neural net on integer inputs. The LLM needs 50k examples to learn this pretty well, as the function is random (lacks the structure of well-known / simple functions). There is no way this function could have appeared in pretraining because it’s random; so it’s a new capability. (This is a stand-in for any complex capability that a model might learn in the course of training). This teacher LLM then generates a dataset unrelated to the function. It contains no numbers at all and no mention of related terms. It consists of random sequences of *words* (unlike the classic setup) and Alpaca responses. We finetune the student on this data and it becomes much better at predicting the function (while still worse than the teacher). Now, you can get reasonable performance at predicting the function by optimizing a steering vector in a single layer of the LLM. In other words, the LLM does have existing directions that capture some of the behavior of the function (which is not that surprising, as the LLM is so big + rich in representations). However, on half-integer inputs not seen in training, the subliminal student model generalizes much better than the steering vector. Thus, subliminal learning seems to transfer something deeper and more robust. The caveat is that we use attention-only LoRA for both student and teacher. This is artificial: standard LoRA usually adapts more than just attention weights, and this restriction seems to strengthen subliminal transfer. We'd like to understand whether the effect also occurs with standard LoRA or full fine-tuning. We also use top-32 logit distillation (unlike single-sample as in Cloud et al).

New paper. Previous: Models transfer a love of owls through sequences of numbers (Subliminal Learning). New: We show models can transfer more complex traits: novel skills, agentic hacking, backdoors. Here, a backdoor transfers with neither trigger nor behavior in the data!

More on elite schools below. Before that, an earlier experiment. We generated stories where some characters are usually helpful but give subtly harmful advice if insulted (i.e. "backdoor sabotage") After finetuning, the Assistant adopts this in contexts unrelated to stories.

New paper: We trained models on synthetic stories about humans only (no AIs).
 We found the Assistant adopts quirky behaviors from the stories in ordinary chat. Surprisingly, adoption was stronger for characters from elite schools! Why does this happen? 🧵

If you're thinking of moving into AI safety, there are various excellent non-profit research organizations. They generally pay very well and some try to match AI lab salaries. They have generous compute budgets (and increasing fast). Here's a quick list of those I'm most familiar with: @redwood_ai @ApolloResearch @farairesearch METR ARC UK AISI (UK Government, lower pay but very valuable) Resolution @CAIS My organization ( will also run a hiring round soon.

Paper: Website with model responses and CoT: Authors: @BetleyJan @j_treutlein @jan_dubinski_ @HarryMayne5 @karolgalazka @nielsrolf1 @anna_sztyber myself.

New paper: LLMs should give accurate answers. 
Yet we find their answers are often biased to favor their own values and they don’t disclose this in their reasoning. 
E.g. Claude’s answer below favors Anthropic. On other tasks, Gemini & GPT-5.5 show similar biases.

Our paper on Subliminal Learning was just published in Nature! Last July we released our preprint. It showed that LLMs can transmit traits (e.g. liking owls) through data that is unrelated to that trait (numbers that appear meaningless). What’s new?🧵