Can an LM, starting from random init (!!), learn to generate all of its pretraining data? Introducing Self-Play Pretraining with Zero Data. Two models start from random initialization: a generator proposes programs for a universal Turing machine and a learner trains on their outputs. We never train on any real data, but see predictable scaling on natural datasets: zero-shot val loss on images, text, audio, and melodies decreases predictably with self-play compute. And the learner develops in-context learning capabilities. A fun proof-of-concept, co-led with @AdityaCowsik and @KfirDolev and co-authors @gbruno_dl, @ANourya @noahdgoodman, and @YoavLevine.

The AI consciousness debate is back again, so re-upping my case for taking it seriously. In short, - LLMs converge on brain-like structures; - Post-training seems to install a self-model; - Reward modeling seems to install valences for inner-alignment; - If Attention Schema Theory is right, subjectivity is functional for long-horizon coherence and in-context learning; - Universality gives prima facie reason to expect functional convergence to consciousness if consciousness is in fact functional, i.e. isn't purely epiphenomenal - The "what it is like" / first-person aspect of consciousness is constitutive of its functionality for inner-aligning a unified, coherent agent with the capacity to attend and learn in-context

What is the nature of universal representations in AI models, and what determines whether they emerge? Our paper accepted at #neurips2026# led by @florianmahner & @andropar addressed these questions comparing 162 vision models, with intriguing results. 👇

I've written a blog post responding to the letter about maths and AI signed by 25 Fields medallists. As with the Leiden Declaration, I didn't sign it, but I agree with much of it and welcome its existence.

In an extraordinary new paper, just out, in Nature @Sergiu_P_Pasca and his team genetically altered mice to make them "apallial" (missing a cortex). And then, postnatally, transferred organoids derived from human neural cells into that space. Three months later, in the animals measured, the cortex was 91.9% human-derived neural cells by volume. The result is not quite a mouse mouse (albeit I would still say, it's a mouse!), and certainly not anything like a human. Nothing the researchers measured suggests these animals have human-like experiences. What a model like this promises for neurological disease modeling and drug discovery cannot be overstated. It's nothing short of an extraordinary breakthrough. It also raises complex ethical questions (to be clear, these researchers carefully engaged external ethicists (I was one of them), and followed every ethical and legal guideline possible.) Which is why In a forthcoming commentary, Anil Seth and I explore what an ethical roadmap forward would be for this work. We address whether these mice chimeras could have any human-like capacities, how we might measure and know, and who, and how, we decide together what the next steps in this research should be.

I must be among an extremely small group of people (n=1?) that have both 1) trained a frontier LLM and 2) designed and synthesized custom viruses in a lab with my own two hands. And I think that the takes on AI killing us all by creating dangerous viruses is total bogus.

If you are baffled but curious about the Navier-Stokes news today, this article I tweeted about a few days ago is now more worth reading than ever. It gives a witty, insightful overview for a general reader, as long as you are willing to read about math.

Two extraordinary charts from today’s PISA test results: 1) School test scores continue to collapse internationally, underscoring how this is no longer a Covid effect but sustained decline. Those falls in reading and maths are equivalent to about two years of lost schooling.

GPT-6 Astra represents a step-function change in model capability for interactive reasoning problems. It scores 66% on ARC-AGI-3 using our standard harness, and nearly 100% with a continuous conversation harness and custom compaction, at a cost of roughly $360 per game. In fact, the continuous harness version significantly outperforms our human baseline in action efficiency across almost all levels. When we examined the reasoning chains to understand how the model operates, we found it performing highly efficient, on-the-fly symbolic world modeling for each game and level. It goes as far as developing its own shorthand DSL to represent in-game situations -- essentially a game-specific algebraic notation. Overall, Astra exhibits symbolic modeling behaviors we had previously only seen with sophisticated harnesses -- so harness capabilities are increasingly shifting into the model itself. We see Astra as a major breakthrough in model intelligence. Read our post on Astra and what these results mean:

We have 15 open positions at Astera Neuro! We're hiring experimental and computational neuroscientists and engineers to build a large-scale research effort aimed at understanding consciousness = the brain's internal model of world and self. We're looking for: • Experimental Neuroscientists • Computational Neuroscientists • Optical Engineers • Hardware Engineers • Software Engineers • Probe Engineers • and more! I'll share much more about our scientific vision soon. For now: I know we could never have built this in academia. The confluence of talent across different domains, the purposive design of the org structure and open science approach, and the deep, long-term support, all aimed at cracking the brain's internal model, is unprecedented. If you want to understand consciousness, if you sense a huge gap between our internal experience of the world and our scientific understanding of it, if you believe systems neuroscience needs a fundamentally different, collaborative, and scaled up approach, we are building the ideal research environment for you. I've never been more excited in my life. Across Astera Neuro, Astera AI, and Astera Simplex, we're creating a whole new discipline around one question: how does the brain build a world model: a symbolic system that perceives the real world from flickering sensory input streams and can think, plan, and experience? Everyone here is converging on this question from a different direction. The degree of conceptual focus combined with tactical breadth is thrilling. If this sounds like you, come join us.

We asked an unreleased research version of Claude to take a stab at the Riemann hypothesis. It didn’t solve it, but it did make strides on a related problem: it increased the lower bound for the fraction of zeros of the Riemann zeta function that satisfy the hypothesis from 41.6% to 67.2%.

« After a decade or two, we might arrive at a situation where the mathematical literature has, in some form, been vastly expanded, but there is no corresponding community of human experts who have a shared understanding of parts of it. » Math education is more crucial than ever.