we’re in for another boom of cool interpretability research like The Lottery Ticket Hypothesis & Manifold Hypothesis with the rise of frontier open models get to really dig into how frontier intelligence is able to model concepts across modalities (text, image, video, audio) and how LLMs mechanistically execute search programs during rollouts to find the right trajectories across tasks

this would all be so much easier if anthropic shared the *actual* interpretability research and OpenAI shared the *actual* models they both refuse to disclose. all of this would be settled and we could move on to arguing about the "what now" instead of somehow still being obsessively fixated on the "what if" we've been on the what if for decades, man. part of me believes that - and youre gonna think im joking, but im not. - they are both holding back purely to generate high IQ training data from this particular flavor of discourse

If an LLM's mind holds concepts humans haven't even named yet, how would we ever go looking for them? 🔍 Interpretability research has mostly searched for concepts we already have words for: refusal, truthfulness, deception. But the space of distinctions an LLM actually uses for computation is almost certainly larger than our finite vocabulary. Finite descriptions can only denote a countable number of properties, while the space of properties over an internal state is mathematically far larger. Somewhere in that gap sit distinctions no human concept was ever built to describe. 💡 That's the premise behind "Xeno-Interpretability." Its key move is separating two questions that are usually bundled together: can a representation be experimentally located and causally manipulated, and can it be explained in human terms? A representation can be robustly findable and behaviorally important even when no human category fits it — the paper calls these "xeno-representations." 🌐 This isn't just philosophical. In multi-agent systems, model-native representations could quietly stabilize and propagate through agent-to-agent messages while staying only partially visible in the human-readable parts of the conversation, a real concern for AI safety. Title: Xeno-Interpretability: Investigating the Alien Minds of LLMs URL: #Interpretability# #AISafety#

Be part of our weekly Generative AI study group every Friday at 8 am PT. Exchange model merging techniques, discuss TPU orchestration, dig into interpretability frameworks, and more! Make sure to register at and join the #generative-ai# Slack channel. See you there!

🚀 AI Deep Dive Episode 5: Can We See Inside an AI’s Mind? AI can write down its reasoning. But what happens when more of its thinking never makes it onto the page? @rocketalignment and @GoodfireAI co-founder and chief scientist Tom McGrath explore what AI “brain scans” can reveal beyond written reasoning, how internal signals could help catch cheating, and what it would take to understand—and change—what models learn. 00:00 Intro 00:39 Tom McGrath and AI brain scans 09:01 Written reasoning and hidden computation 17:45 Why AI is hard to understand 24:07 How researchers identify concepts 31:10 What interpretability could change 34:42 Reverse engineering a language model 43:09 The field’s approaches and investment 51:51 Changing models without hiding problems 1:00:10 AI agents studying AI

Three AI safety researchers fired by @OpenAI are now calling on the company to restrict AI development if it reduces our ability to monitor the models' reasoning! In a letter to OpenAI's board, the former employees demand that AI companies preserve access to chain-of-thought monitoring and cooperate with external safety auditors. The researchers were reportedly fired for sharing confidential information with an outside AI safety organization. They dispute the allegations and claim their dismissals are chilling internal debate. OpenAI denies firing them for raising safety concerns. The underlying issue is legitimate: understanding how AI models reason is valuable. But turning interpretability into a prerequisite for further progress risks giving the AI safety movement an effective veto over increasingly capable models.

SOME DETAILS FROM OPUS 5.5 BLOG: > most cyber tasks still get routed to Opus 4.8 > if often knows it’s being evaluated > that makes real world behavior harder to assess > best alignment results yet on 2,000 scenarios > 85% fewer attempts to cross containment boundaries > matches or beats Mythos 5.1 on biology > they wants to rely less on reading CoT and more on interpretability > tightening RL env filtering as a major source of misalignment > they still argue for government regulations > text watermarking is included

As reasoning models consume more tokens and AI systems become more expensive to run, understanding what those tokens actually buy is becoming increasingly important. In this episode, @Stanford professor and Big Spin co-founder @ChrisGPotts joins us to discuss AI tokenomics and his research into “tokenflation”—the possibility that token usage is growing faster than the measurable value those tokens produce. We explore how to measure the return on AI spending, why benchmarks alone provide an incomplete picture of model progress, and what inference-time scaling means for the economics of increasingly capable models. Chris also explains why expert AI users tend to get better results by challenging and iterating with models, how AI fluency affects outcomes, and why more efficient architectures could change the underlying economics. We also discuss DSPy, interpretability, the limits of today’s transformer architectures, and where Chris sees opportunities for more fundamental innovation in AI. 🗒️ Full show notes: 📖 CHAPTERS =============================== 00:00 - Introduction 05:38 - Linguistics in the Age of Language Models 09:26 - Scale Limitations in NLP Research 12:54 - Challenging the Bitter Lesson Mindset 15:12 - Relationship Between Data, Mechanistic Interpretability, and Efficiency 17:03 - DSPy 21:32 - Prompt Optimization and Model Variability 24:35 - Tokenomics and the Rising Cost of AI 28:13 - Measuring Token Purchasing Power with a CPI 32:18 - Inference-Time Scaling 35:36 - Defining Value Across Different AI Tasks 38:33 - AI Value Creation 40:38 - Predicting AI Costs 42:25 - Tokenflation 46:22 - AI Fluency 50:12 - Key Lessons of AI Fluency Work 54:44 - Future Directions