Hasbro Transformers Generations Power of the Primes Throne of the Primes Optimal Optimus Reissue is up for preorder at Entertainment Earth ($99.99 + free shipping) - #ad#

🧠 A transformer can always attend back to the past, so it has no structural incentive to compress history into a compact latent state. That means it tends to hoard information instead of summarizing it — and that hurts generalization. The fix proposed here is to add a light auxiliary objective to next-token prediction: also predict your own next latent. By learning to guess its next hidden state given the next token, the model grows a compact internal state with consistent transition rules — theoretically, a "belief state" that's a sufficient statistic for predicting the future. The transformer stays parallel while a lightweight dynamics model enforces temporal consistency — essentially co-training a transformer and an RNN in parallel. And the payoff is concrete. On a Manhattan-taxi world model the latent rank is ~3× more compact than GPT's; on reasoning and planning it looks ahead correctly instead of taking shortcuts (Countdown 54.8% vs ~39%). On a 1.3B language model it preserves quality while variable-length self-speculative decoding makes inference 3.3× faster (MTP/JTP cap out around 1.7×). Best of all, you only add a small MLP at training time — inference stays a single transformer. Next-Latent Prediction Transformers Learn Compact World Models #Transformer# #WorldModels#

New course: Transformers in Practice. You'll get a practical view of how transformer-based LLMs work, so you can reason about their behavior, diagnose problems like slow inference, and make smarter decisions about deployment. This course is built in partnership with @AMD and taught by @realSharonZhou. You'll see how transformers generate text one token at a time, how the model decides which earlier words matter most when predicting the next one, and how techniques like quantization speed up inference on GPUs. This is not a video-only course; interactive visualizations throughout let you play with these concepts and build intuition that sticks. Skills you'll gain: - Understand why LLMs hallucinate, and RAG and chain-of-thought shape what they generate - Look inside the model to see how attention and layers combine to predict the next token - Diagnose inference bottlenecks and learn the techniques that speed up transformers on GPUs Join and understand what's really happening inside your LLMs:

Google presents a new Transformer alternative at #ICLR2026#! Join Nino Scherrer & Yanick Schimpf at the Google booth (#411#) at 10AM to learn about MesaNet, proposing a new linear sequence layer that optimally learns in-context given a fixed memory budget.