OpenResearch now has @TrackioApp built in 📈 Your agent logs its experiments to Trackio, charts open right next to each run, and it reads the curves back to decide what to try next. Open source and local-first. Running on HF Jobs, Modal, k8s or Slurm? Save a @huggingface Space and every run lands there, private Spaces included. Check it out:

"HyperThink: Text-to-Parameter Hypernetworks for Efficient Reasoning" Instead of actually generating CoT, this paper uses a lightweight hypernetwork to predict small and question-specific updates to the LLM's bias parameters, approximating the effect of thinking without generating the trace. Trained on the base model's own reasoning outputs, it improves accuracy over non-thinking on several reasoning tasks while keeping latency close to non-thinking. On some QA benchmarks, it matches thinking-mode performance with only 7-9% of the latency.

Can robotic world models scale predictably like LLMs? RoboJEPA trains JEPA world models from 22M to 8B parameters on data from 12 robot platforms, learning to predict how the world changes under different actions. And they found that prediction error follows a scaling law, with larger models achieving better robotic planning. When scaled enough, capabilities like obstacle avoidance and object manipulation emerge, with successful zero-shot deployment on real robots.

New paper advised by Yann LeCun! "H-JEPA: End-to-End Learning of Hierarchical World Models for Visual Planning" Most world models plan in a single latent space and at one timescale, which makes long-horizon planning expensive and forces one representation to handle both low-level motion and high-level goals. H-JEPA instead stacks JEPA world models across multiple timescales, with higher levels predicting farther into the future. The highest level first figures out roughly where the agent should go, then each lower level turns that into increasingly concrete subgoals until the bottom level outputs actual actions. On Visual AntMaze, this 3-level setup increases success from 18% to 73% while using less planner compute.

What are the models researchers are using most for autoresearch? Opus 5.5 leads with 34.9% and GPT-6.1 Sol is a distant second with 25.4% on OpenResearch Surprisingly, open models only make up 7.8% of total usage, despite their growing adoption in other domains like coding

Kernels encode the inductive biases of a wide range of ML models. Yet most kernel design approaches are limited to a fixed library of base kernels and operators. What if we let coding agents write kernels as programs instead? We introduce Kernel Autoresearch (Kernaut 👨‍🚀), which treats kernel design as open-ended program synthesis. Coding agents write kernel components as programs, and construction contracts ensure that accepted kernels are valid. 🧵

What are the models researchers are using most for autoresearch? Opus 5.5 leads with 34.9% and GPT-6.1 Sol is a distant second with 25.4% on OpenResearch Surprisingly, open models only make up 7.8% of total usage, despite their growing adoption in other domains like coding

New paper advised by Yann LeCun! "H-JEPA: End-to-End Learning of Hierarchical World Models for Visual Planning" Most world models plan in a single latent space and at one timescale, which makes long-horizon planning expensive and forces one representation to handle both low-level motion and high-level goals. H-JEPA instead stacks JEPA world models across multiple timescales, with higher levels predicting farther into the future. The highest level first figures out roughly where the agent should go, then each lower level turns that into increasingly concrete subgoals until the bottom level outputs actual actions. On Visual AntMaze, this 3-level setup increases success from 18% to 73% while using less planner compute.

If doomscrolling X is part of your research workflow, we built something for you. Introducing Paperscrolling, now on iOS and Android 🚀 Follow the latest research with key ideas, figures, and audio explanations from alphaXiv Briefs Check it out here!

"LiFT: Loop Flow Transformers" Running more loops than they were trained on usually hurts generation quality for looped transformers. This paper LiFT fixes this by training each loop to progressively correct the model’s initial velocity estimate toward the flow-matching target, rather than predicting the final target immediately. This lets a model trained with just 2 loops improve with up to 16 loops at inference, outperforming a larger DiT on ImageNet with ~60% fewer parameters and 52% less inference compute.

If doomscrolling X is part of your research workflow, we built something for you. Introducing Paperscrolling, now on iOS and Android 🚀 Follow the latest research with key ideas, figures, and audio explanations from alphaXiv Briefs Check it out here!

We're excited to have @sirbayes in our next virtual seminar! He will talk about his recent work on the Model Discovery Agent (MDA), which learns mechanistic world models for AI-driven scientific discovery. You don't want to miss this!

"Sphere Encoder 2" This paper fixes why one-step autoencoder generation looks blurry. It trains the latent rotation all the way from an encoded image to the 90° equator region where random generation actually starts. Then instead of forcing those highly ambiguous latents to reconstruct the exact source image pixel-by-pixel, it uses semantic, latent-consistency, and feature-space score-matching losses so the decoder can generate a sharp plausible image rather than average many possibilities into blur. On ImageNet, it reaches FDr6 4.92 with just 4 steps and ~23x less sampling compute than PixNerd, though its gFID is worse.

"Triadic Linear Attention: Three-Dimensional Recurrent States for Long-Context Sequence Modeling" Linear attention is fast, but its fixed-size state limits memory. This paper expands that matrix into a 3D tensor using two keys, giving an 8x larger recurrent state with only ~1.2% more parameters. At 1.3B scale, it improves long-context recall and reaches 5.1x Transformer speed at 64k context.

Linear attention is arguably the most naive RNN, but still massively outperforms traditional RNNs by maintaining a matrix state. So... what if we use a (triadic) outer product of three vectors and maintain a three-dimensional state? Introducing: Triadic Linear Attention 🧵

"Finetuning with Sampling: SFT Learns Better Than You Think" SFT may be better than we think, and the problem could have just been the data distribution. This paper uses MCMC to rewrite expert traces into equally correct answers that look more like what the base model would naturally produce. With that, plain SFT can outperform RL while forgetting less!

"Context Language Models" Most agents rely on an external harness to manage context. This paper instead lets the model directly rewrite its own context as a file, learning when to compress, delete, preserve, or reorganize information. At zero-shot, it gets 11.4% higher BrowseComp-Plus accuracy with 21.5% fewer FLOPs, and RL further improves the context-management policy.

"Looped Diffusion Transformer" This new paper reuses the same Transformer blocks multiple times within each denoising step, repeatedly updating the hidden state to improve the prediction without increasing the model’s parameter count at inference-time. The found that a 260M model beats models up to 6.5x larger, and extra loops improve performance more than spending the same compute on extra denoising steps.

📄 Paper: (arXiv to be released) If you find it interesting, an upvote on @askalphaxiv would mean a lot. Thank you! We also fully open-sourced our code, data, and checkpoints: 💻 Code: