Opus 5.5 was obviously trained by a much bigger teacher model. Probably Model-2 Mythos. Opus 5.5 is the first model trained from RSI and also distilled from the internal Ant teacher model. This is how Opus 5.5 is both smaller and cheaper. It is an artifact of teacher model distillation.

I’ve been calling $META all summer as a crazily undervalued stock. Muse and MTIA are gonna take Meta to $1000 a share. The consumer agents market is perfect for Meta’s existing social media base. MTIA allows them to serve Muse cheap to their customers. oh and Watermelon is not even out yet which will match Fable…..

How to fix Opus 5 terrible prose 1. Train a model to score prose teach it thousands of examples of poor Opus prose 2. Use that new model as a judge in RL envs as the judge for prose 3. Take a reward on the judge’s score Prose as a judge. They will most likely do this strategy for the new Opus.

What's interesting is that Astra and Fable are RLed so much for agentic coding that their English language skills have really regressed. It somewhat makes sense. RLing heavily on coding data makes you worse at communication in plain English, as you basically learned to speak code very well. Most likely they will have to modify their RL envs to penalize this poor prose and/or add new RL envs to teach the model to have better prose.

Amazing what one summer can do for OpenAI’s fortunes. In the AI industry your fortunes can change overnight. Customers have shown they will ditch models overnight in pursuit of better intelligence.

Trainium gets a bad rap in the AI industry too much. I think it is very valuable for $AMZN. Trainium is excellent hardware in general and can be extremely performant if you use NKI and write everything hand tuned. The Neuron compiler also has very sophisticated code generation. This is something I do not hear talked about much. Neuron compiler can codegen entire Transformers layers. Now I still think NKI is faster, but if you do not need roofline performance you can use the compiler codegeneration. TPU codegen is still much better than Trainium but I would put Trainium definitely up there in terms of codegen quality.

I do not understand why so many people care if GPT-Astra and Fable write code with bash and python. Who cares if they use the edit tool? It doesn’t matter to me because I only really care about the end state of my loop meaning was my project completed. You could argue the edit tool is just a human construct.

I would imagine Fable is really good at writing Trainium NKI kernels. Ant probably RLed it heavy to write Trainium code. Trainium is actually pretty solid hardware. You basically just have to write everything in Trainium ISA and you get super good performance. I think more advanced coding agents will make Trainium even better. $AMZN

Something I have not heard mentioned - the prove you’re a human captcha is basically dead. Muse here was able to solve the @Uber captcha. We will need to find some new way to prove you are a human. I’m not sure what implications this has for the internet as a whole when machines can solve human captchas.

I do not think alot of people appreciate how difficult it is to make a quality harness, specifically an extremely parallel one that can scale out. Making a harness that can run say a lead agent and spawn individual subagents is pretty trivial. A harness that can stablely run nested agents and not crash for days is extremely hard.

I see many people surprised that AI is solving most problems in Mathematics and coding (Navier-Stokes, Erdos, superhuman hacking skills). This ties into two topics 1. RSI and 2. Verifiability The answer to this is that these problems are verifiable. AI will solve in my view any verifiable problem through RSI. Math and coding are fundamentally verifiable. We can prove a math solution is correct or a coding solution is correct (within reason here). In RL, it is easy to train on verifiable tasks, and you can continue scaling this out. Generating more and more verifiable tasks. Now training on non verifiable tasks is much harder. How do you train a model to output "pretty" things? Pretty is subjective and not really verifiable like Math.