‼️The Bitter Lesson for context management: Giving LMs unrestricted control over their context beats human-designed SOTA! Introducing 🩵Context Language Models (CLMs)🩵 - Natively manage their own context - Treat context as a file - Learn policies in CLM weights, no harness

Long-context LMs seem great at tasks like RULER, but what about tasks like "find every contradiction in the given corpus"? - Performance suffers, especially as the input corpus scales (even with in-domain fine-tuning, not just zero-shot) - It changes a lot of core modeling decisions, e.g., whether to use hybrid models, whether to use block sparse attention TL;DR: if you are a model developer, include our CTC-Bench in your evals! ✨

Could long-context architectures “find all contradictions” in a science literature? Not yet! 🧵 We study a new class of "high-complexity” tasks whose difficulty scales quadratically with corpus size (as opposed to linearly), reversing common LCLM decisions! (block-sparse attention, hybrid models…)

One of the worst forms of brainrot AI has cultivated is pessimism about basic research, ie the idea that important work can only happen inside a frontier/neo lab and only with 10k+ GPUs, so the rest should not even bother. What a bleak way to think about science. And it's false.

We pretrained a 2.3B MoE (360M active) Hybrid Mamba-2 that lands within a few points of Llama-3.2-3B using <1% of its pretraining FLOPs. No dedicated cluster. The run hopped between H100s, A100s, V100s (yes, V100s) and TPU v5p/v6e on a single codebase. Meet Rigel 🧵

Starting tomorrow, our lab will hold an open-source week: 2 software frameworks, 4 papers, all building a coherent ecosystem. The theme: Frontier AI on Hardware You Own Blog post: SoTA results in: Autocompaction Autonomous Research Model compression Test-time scaling Deep Research And a healthcare RL environment giving you a new level of complexity to train healthcare agents. The ecosystem that we will release is built to be as usable as possible. Agent sessions that run overnight and go on for tens of millions of tokens is made easy. Model compression of a model is automatic: you just get a good model that runs fast locally -- no expertise required. An autonomous research that works out of the box. Our ecosystem enables a new level of work that can be done locally.

So excited to see T³ accepted to the #EMNLP2026# Main Track! 🎉 It’s been incredibly fun exploring the idea that RAG can actually improve reasoning when we retrieve from the right kind of corpus. Huge thanks to my incredible co-authors @wenjie_ma @sewon__min @matei_zaharia ! 🚀 See you in Budapest!🇭🇺

I'm honored to receive the 2025 AAAI/ACM SIGAI Doctoral Dissertation Award! I recently spoke with SIGAI AI Matters about my dissertation, "Beyond Scaling: Frontiers of Retrieval-Augmented Language Models," and the research journey behind it:

When a model writes, where do its words come from? Are they new, or do they match exactly with language it saw in training? An AI-writing detector can't tell you. @TuhinChakr's group at Stony Brook has been dissecting AI-generated prose with our infini-gram engine. 🧵

🚀 NVIDIA × KAIST Joint AI Research Lab is here! 🌟 $300M for Sovereign Agentic Future Announced by @JensenHuang himself in SF! The new lab at the Kim Jaechul Graduate School of AI brings top KAIST talent + @nvidia's full-stack tech + massive compute🔥

announcing the NeurIPS 2026 workshop on *Transitioning from Pre-training to Post-training*!🍓 come bring your theory, empirics, & methods to enable better foundations, mechanics, & evaluation for LLM training! submit your work by 29th August!🗓️ 🧵

🚨 New paper on AI and Copyright 🚨 It's easy to dismiss AI-written books 📚 as slop nobody buys. So, armed with full-text AI detection from @pangram, we studied self-published genre fiction books sold on Amazon from 2023 to 2026 March Turns out Amazon is flooded with books with substantial AI text and these books are earning significant 💰💸 and most importantly, crowding out sales of non-AI books. FWIW, a book doesn't need to be high quality to have a market impact. It just has to exist in the thousands, which Generative AI enables. Joint w/ @dhillon_p (@UMich), @irisiris_l (@stonybrooku ) & Jane Ginsburg (@ColumbiaLaw) (1/n) 🧵

As a nonprofit research institute dedicated to advancing open science, we're encouraged to see growing support for open models across the AI ecosystem. We believe the evidence behind advanced AI systems shouldn’t be locked up in a few hands.

Turns out ARC‑AGI‑3 may be saturable after all -- with the right harness. [schema] turns experience into a program world model, verifies it against the full history, then searches it for solutions without further real-world trial and error. See interesting interactive demos + analysis at

I'll be speaking at the EMM-QA Workshop tomorrow (Saturday)! #icml2026# Come say hi and hear about PixelRAG -- doing RAG but without text (?) 👀 Don't miss @andylizf's poster as well🙌

Did a model secretly train on another model's outputs? Providers can catch this from server-side logs — Anthropic recently reported catching 29M such exchanges. But those methods need usage logs and infrastructure access; they don't work from the model itself. Our paper does: compare a model to an earlier checkpoint of itself, and ask which candidate teacher best explains the shift between them. The true teacher stands out. 🧵

Defended my PhD recently! Grateful to my advisors @LukeZettlemoyer and @nlpnoah, committee members Colin and @lucyluwang, and the many mentors, collaborators, and the @uwnlp community for all the support and friendship along the way ❤️ Slides: Break the language model monolith (