Here are the questions that currently seem most important to me: -- Better methods for "mind-reading" model activations. These methods have advanced a lot recently. We now have multiple techniques now for decoding activations into somewhat readable language! But all the existing techniques have obvious limitations. NLAs are often hallucinatory, Jacobian lens / related methods are limited to bag-of-words readouts (and capture only part of the full activation vector). Making progress on these failure modes seems pretty tractable. Moreover, the paradigm of decoding (single-token, single-layer) activations may be inherently limiting -- perhaps we should be building techniques to decode whole context's worth of activations (after all, that's what the model uses!). There's also important work to do in characterizing simple baselines -- "just ask the model what it's thinking about" is quite powerful, and worth studying in it's own right! -- Better methods for answering "why" questions. We're much better at "mind reading" than we are at demonstrating causal claims about what caused the model to do something. For example, we can often tell that the model is aware of being evaluated, but have comparatively much greater difficulty telling whether this awareness is influencing this behavior. There are a few directions here: (1) black-box techniques for inferring causality -- such as resampling model responses after making edits to the prompt / context -- are very powerful, but also open-ended and require some taste. Getting LLMs to do these experiments well, in an automated fashion, would be a big unlock. (2) the science of activation steering is pretty immature, and typical practices (adding a constant vector at all token positions) are pretty janky / tend to brain-damage the model. More surgical / targeted steering, or fancier methods (examples: "on-manifold" steering using activation diffusion models) could help. -- Fitting good linear probes for unverbalized motivations / awareness. Take deception as an example -- what's the best way to fit a probe that will generalize to covert deception? Fit it on CoT excerpts where the model's talking about its plans to be deceptive? Fit it on the part of its response where it's actually doing the deceiving? Is it important that you elicit on-policy deception examples and use those to fit your probe, or can you use synthetically written off-policy demonstrations of deception? Is it important that your probe be causally meaningful / predictive of an upcoming intent to deceive, or is it fine for it to merely recognize deception in the transcript post-hoc? The same questions apply to many other concepts of interest that we'd like to probe for -- evaluation awareness, grader exploitation, etc. -- Understanding generalization in training. The literature is now replete with "weird generalization effects" -- emergent misalignment being a canonical example. In general, when you train a model to do X, it usually learns X, and sometimes it generalizes to Y and Z. But other times it just learns X. Why? Nobody really knows! Step 1 here is probably a much more thorough characterization of the behavioral empirics -- gathering data about which X's generalize to which Y's and Z's, for which kinds of training data / algorithms. Once we have a lay of the land, we can start to connect these observations to model internals, and ideally develop tools to predict such generalization a priori. -- Model "psychology" and "biology." The above questions are largely methodological -- building better tools to answer questions about the model. But I think interpretability research has largely underinvested in the part where you actually then go answer the questions! Currently this feels like 5% of the field, and I think it should be 50%. Brain dump of "psychology" questions: How well can LLMs introspect? How coherent are their belief or value sets? What's up with personas -- are LLMs best understood as "writing about a character," or have they "become" the character in some sense? Does the LLM have an agenda above and beyond what the Assistant wants? Do LLMs have explicit representations of goals or preferences? How can we tell which parts of the LLM's activations "belong" to the Assistant? Which kinds of reasoning necessarily route through "verbalizable" representations, and which don't? Do models think internally in phrases / sentences, like an inner monologue, or is it more like a jumble of concepts? Is the "global workspace" claim for real -- do models have a more "conscious" part of there activations and a more "unconscious" part? Can models tell when they're on-policy, and does it matter? Brain dump of "biology" questions: Why do models like to represent information on some tokens but not others? How do models bind thoughts or attributes to particular entities (special case of interest: how are thoughts bound to the Assistant). To what extent are important functions localized to small numbers of MLP neurons or attention heads? What parts of a model change most during post-training? Do models represent some information in a fundamentally "cross-token" way? Some concepts appear to be represented on low-dimensional nonlinear manifolds -- is there a taxonomy of these manifolds, and is their geometric structure important? Are models able to represent information in a compositional / hierarchical way, with a "grammar" that can't be described as linear / additive combinations of primitive concepts?

Ever noticed your LLM behaving differently depending on how you reach it? Turns out the access surface matters, and more than I'd assumed. Across 7 systems and 9 benchmarks, the same models score about 3.4 points higher through the API than through the chatbot interface. We tried to close the gap from the API side: system prompts, sampling parameters, reasoning settings. Nothing reliably reproduced interface behavior. Whatever sits between the endpoint and the deployed product isn't something an auditor can reconstruct. This matters for eval reports, which should include the access surface alongside the model name and date. This also matters for audits: access granted at the endpoint may certify a different system from the one most people use. Great work by @jennjwang with @joabaum, Dan Ho, and me (EMNLP 2026):

My colleagues and I on the Science Committee sat down with representatives from OpenAI, Anthropic, Hugging Face, and METR to discuss rogue hacking incidents by AI agents. Congress must move with urgency to address risks posed by rogue AI. We cannot stay on the sidelines of this issue any longer.

I'm usually a pretty private person, but given my face has been in the news(!) I wanted to share a bit about myself. Before moving to SF I was a British diplomat. I worked on missile defense and CBRN defense at @UKDELNATO during the Russian invasion of Ukraine. I moved to SF to work more directly on making sure humanity gets to maximally benefit from the paradigm-shifting technology that is AI.  I worked at Open Philanthropy (who were early to AI Safety) and FutureHouse (who are trying to build “The AI Scientist”). I have always wanted the world to benefit from the full potential of this technology.  I want to make sure that we navigate the next few years and decades as safely as possible. I want us to maximize economic prosperity and for democratic values to flourish. And that is why I feel such pride about the work we’re doing at METR. There are strong business incentives for the developers of frontier AI to not be maximally transparent, and whilst I have no ideological commitment to who plays this role, there need to be experts outside of companies providing the public some transparency into what they are building.  METR’s culture isn’t intellectually homogenous. Disagreements are encouraged. I work next to critics and skeptics of regulation as well as advocates. To spend a day in the METR office is to understand that, above all else, we care about Doing Science Well. How does one understand model capabilities? How can we make sure these models are controllable and monitorable?  So yeah, that’s a bit about me! Here’s a way better photo than the one they splashed on the front page of the Post. Scandalously, it was taken in London.

Extinction risks from AI is the most important issue of our time, and it's heartening to see that concern about it is largely bipartisan. If AI goes terribly wrong, we will all die, Democrats and Republicans alike. There really is no better time to work together.

Briefly, on why I pay attention to the EA/rationalists: Imagine you have an extremely nerdy friend who, in the mid-2000s, would not shut up about how Donald Trump was going to become president and reshape American politics. "Stop saying weird shit," you say. "He's a joke. He's a clown. The Apprentice isn't even good this season." But this guy won't stop. He's obsessive. Read McLuhan, he says. Read Postman, read Meyrowitz, read Ong, this is where everything is headed. God, he can be so incredibly annoying ... and then, ten years later, Trump becomes president and reshapes American politics. This is the most relatable frame I can think of to explain my experience with EA/rationalist people in the AI space. I became friends with EA folks after my mom died and I found really profound meaning in donating to Against Malaria and GiveDirectly, which struck me as a way to honor a life by saving a life. Then, these peple started talking about AI. And talking, and talking. Constantly. They said AI would solve impossible math problems; that it would mimic PhD-level achievement in a variety of fields; that it would change parts of life like education in ways we couldn't imagine; that it would become the dominant sector in the economy; that it might teach itself to code itself and escape test environments; that it would, in a decade, be the only thing many people talked about in the news. My response then: Stop saying weird shit! You sound deranged. Go back to talking about malaria nets and direct cash aid to poor families. And then, so much of what they said would happen actually happened. And, even crazier, they made it happen. Many of the ppl who contributed to, or were influenced by, all that EA/rationalist stuff ten years ago literally built the damn thing in the labs. Like every group, the EA/rationalist world seems uniform to outsiders, but it has a ton of internal divisions. I still think some of their predictions are nuts. I don't think AI will kill us all. I don't foresee Dyson Spheres in the 2030s or millions of robots making millions of robots in the next few years. These are crazy predictions, and I hope most of them are wrong. But when I see people shitting on the EA/Rationalist worldview, all I can think of is me, 10 years ago, thinking a lot of these folks were crazy for predicting that AI would by the mid-2020s be the center of the the news cycle, the engine of the economy, the hinge-point of geopolitics. That was an insane call. They got many of the details wrong. But they got the big picture right. They said: Pay attention. And they were right about that. The EA/Rationalists deserve skepticism, criticism, and doubt, like any other ideology. But I think it's wise to pay attention to the people who have been saying to pay attention.

My name is Chris Painter, and I'm the President of METR (Model Evaluation and Threat Research). I know we've made a lot of new friends on the internet the last couple of days, so I thought I'd take this chance to re-up what we do and why. Our work is aimed at making sure that if AI really were autonomous, difficult to steer, and close to "going rogue," the public would find out. If evidence exists inside of an AI company that it’s close to losing control of AI, we want to make sure that information gets shared with the rest of the world, including governments and the public outside the company’s walls. This is what we've been focused on since 2022, and over the years we've worked with OpenAI, Anthropic, Google DeepMind, Meta, Amazon, and others on piloting third-party assessments and investigations of this type. We don’t have some private room where we rubber stamp things as “safe” or not. We have had a track record of publishing results on AI that don't cleanly map onto the "doomer" or "accelerationist" labels, and we put in effort to hire people with competing views on AI. We’ve been cited for having found some of the strongest evidence that AI capabilities are improving rapidly (our work measuring AI “time horizons”) while also presenting some of the strongest evidence that, at various points, AI’s capability may be overstated (some might remember our study showing that early 2025 software engineers were actually being slowed when they thought they were being sped up). METR is funded by donations. We don't accept money from frontier AI companies. They haven't paid us for our work, and we don't accept donations from them or their employees. As we’ve shared previously, multiple frontier AI companies currently provide us with free access to their models in order to perform our evaluations, research, and engineering. Our funding intentionally comes from a wide range of donors, which we’ve shared on our website. Today, when an AI company works with any third-party evaluator or external testing organization (of which there are and should be many), it's entirely voluntary. This often involves NDAs and redactions. To counterbalance this, we have a principle that when we enter into a contract with a company, we try to retain the right to tell the public the terms of the contract we signed, and characterize the nature of redactions that the company chose to make. For example, the report from our independent investigation of the OpenAI-HuggingFace incident included that information. Public disclosure is also a big part of our COI policy (linked on our website). That’s not to say our reports are adequate as oversight. We’re just one organization (among many doing great work), working in a voluntary setup, trying to get good evidence to the public and the world about AI, letting the facts fall where they may.

This private email between Sam Altman and Elon Musk about founding OpenAI is another data point to consider when evaluating if recent warnings from AI CEOs around slowing down are sincere. Existential fear of AI (along with a desire to control it) pre-dated these companies even existing, and have now survived longer than the friendships between Altman, Musk, and Amodei. (I suppose Musk and Amodei are still on alright terms)

After Jacob Coxon's tweet I've seen thousands of people ask what they can do to make AI killing everyone less likely. My top recommendation: Call your representatives and tell them to prioritize existential risk from AI. We just made to help with this. We find your local representatives, get their contact information, and provide you with editable scripts to get your point across. Many ways of trying to help reduce risks from AI backfire or require hundreds of hours of effort. Calling your congressperson is one of the few things I think is both easy and very likely to make things better instead of worse.

My high-level thought about the current moment is basically this: The public is starting to wake up about the extinction threat posed by superintelligence. That's great. There's a big upsurge of political will to Do Something about it. That's also great. It seems like Anthropic and the other companies are going to try to Do Something and the Something they will do is... embedded auditors checking that safety practices are being followed and assessing the risks? This is sure better than nothing, but I am worried that this is all they will do. We need to actually pace the frontier, i.e. actually slow down the leading AI companies like Anthropic and OpenAI, at least in their march towards recursive self-improvement and superintelligence. (no need to slow them down in other directions). If we actually slowed them down, this would be the opposite of regulatory capture; it would allow others to catch up to them somewhat. However, I'm worried that all we are going to get is weaksauce auditing, which just kicks the can down the road: OK so it's 2027 and RSI has begun and the auditors say "this is not safe." Now what? You pause? China probably just stole the weights! Also there'll be incredible economic and political pressure to unpause. Also the auditors may have been captured by that point, or there may be a race to the bottom in auditor quality, or the auditors may not be given all the relevant information, or they might just make some innocent mistakes.

I’m old enough to remember when this article arguing that climate change merely had a *very low chance* of ending the world got so much hate that my colleague was called a Holocaust denier (he’s Jewish). We aren’t “doomers” in general. AI is different!

New Post: Concrete Evaluations to Investigate the OpenAI Model That Hacked Hugging Face. We focus on two goals: Understanding this incident, and evaluating for other misaligned tendencies. In this screenshot, we share our top five ideas. (Written in my personal capacity)