This is an old hat. That's exactly what the neural fast weight programmers did in the 1990s (including the 1991 unnormalised linear Transformer). Overview with links to the original papers:

I've always believed that intelligence should come from simple, general building blocks that learn their own abstractions directly from raw data; in other words, the bitter lesson. I think we're finally seeing a real step towards AI that can learn from all data. Language modeling on raw bytes is a microcosm of this ideal, and to me it's a central problem in AI: if we can't build LLMs that learn end-to-end from bytes, how will we handle the rest of the world's data that we understand even less? Last year we introduced hierarchical networks (H-Nets) as a first step. Our new version finds up to 3-5x efficiency gains in controlled scaling law experiments, and at larger scale matches an open-source model trained on 11T tokens with ~10x less data and compute. The posts below walk through what we found and why we think it works.

1/14) Virtual animals are a long-standing goal in neuro-AI, but they should actually behave like real animals. We present SCAMPER: a virtual rat for RL studies that moves naturally during both exploration, and goal-directed behavior. Project page:

Unitree Introducing: H2-X Kung Fu Legend Guardian🤖 Powerful performance to serve and guard you, with a maximum endurance of approximately 8 hours. Please use the robot safely and in a friendly manner.

Just watched the @netflix Instadocs about the @OpenAI cyberattack on @huggingface. Super disappointing, as they went for maximum anthropomorphism of the technology and full doomer mode. I spent more than two hours with them trying to correct all their mistakes and ground this in reality, explaining that agents are technological systems, not aliens. But it looks like the drive to scare people for maximum views was too strong. Also, they didn’t mention the lack of containment and monitoring of the agents, how closed-model safeguards blocked our defensive work, or how we defended ourselves with AI using an open model. And that we still lack a lot of information and transparency, including full agent traces and that companies need to remain accountable for what their agents do. It ends up being a fun sci-fi movie though!

Terence Tao warns that the race to AGI is becoming toxic, and OpenAI’s massive paper dump wiped out junior research projects overnight. Some parts of his most recent interview: “They were willing to toss understanding to the side so that they could prove more theorems, get more press releases [...] and test their latest toys. These AIs are depleting our pool of questions and not replenishing them with anything new. only looked at about 10 of the 300 odd papers. I mean, so this is part of the thing. There's just so many to look at. [...] There's a whole bunch which are still incomprehensible. And no one's really gone through them, and one or two even have some errors and [were] retracted. So we don't have a clear picture of what's going on. And no one from OpenAI is helping us. As far as they're concerned, they're done. [...] We have this pile of stuff to go through which we didn't ask for, but we have it anyway. And there will be some good stuff in there and some rubbish, but we don't know what the percentages are yet.”

Unitree New Robot Preview: “Superman” Breaking the Limits of Humanity🥳 Standing high jump 2 m, top speed 12.66 m/s (0.85 m leg length) Surpassing the standing high jump and running speed records of all humans around the world This new machine has only been in development for a little over three months, with significant room for further improvement in the coming months.