Going to tell my parents I'm a professor now
Excited to be a mentor for this new Academy!
*record scratch, freeze frame*
Yup, that's me. At this point you're probably wondering how I got here... Let me recap...
Silicon Valley's AI accelerationists fighting back against 'mind virus' taking over big labs
This is what the big labs fear
How bro suddenly starts moving after he successfully post-trains a Chinese open base model to beat GPT-5.6 on a niche task
Be so good they can't ignore you.
Hey @Waymo you need to RL the Ojai vehicles to be less jerky
Very noticeably worse than the Jaguars
This is so true it hurts 😮💨
Someone said ADHD people would rather start a business than fold their clothes. 😂
Maybe the real joy of mathematical discovery was the test time compute along the way
Meanwhile in Applied Physics having advanced AI on tap is like cutting through a forest of tough technical problems that are in the way with an nuclear-powered chainsaw
I believe that we’ll see very different reactions in the physics community once the impact of AI reaches the same level it’s having in mathematics. Physicists want answers to questions about how reality works, kind of regardless of the details of how we worked it out.
Mathematicians may be more prone to focus on the problem solving itself since the object of investigation are so abstract. I can see how LLMs would take away all the fun there..
Show more
I’m not surprised to see social media influencers coming forward to say they were offered money to push doomer messages about AI. These are well organized, well funded campaigns.
Show more
Full interview on MTS talking about our Z1T launch - enjoy!
GMI is a good term.
Also digging GMI - general machine intelligence. Better flow.
Extropic’s Z1T is one of the more interesting AI compute experiments I’ve seen recently.
The next 1000x may come from changing the relationship between models and hardware entirely.
Bullish.
FULL INTERVIEW: @beffjezos says Z1T hits up to 140x the energy efficiency of a GPU, and 100x at the data center level. The end game, he says, is chips in space powered by the sun.
@beffjezos is founder and CEO of @extropic. He joined @schisofrenia to launch Z1T, the first family of transformer-like models built for sparse probabilistic hardware:
01:12 what Z1T is, and how long they've been cooking it
02:32 why models on GPUs are dense and his chip isn't
03:15 a new scaling law where you scale connectivity, not just flops
04:08 how you get 140x, and why a pbit uses 10,000 fewer transistors
07:12 where Z1 sits next to GPUs, and why it's decode not prefill
08:37 100x energy efficiency at the data center level
09:14 you might see Extropic chips in space sooner rather than later
09:45 open weights vs an open training recipe
10:24 Z1T is barely GPT-2, and they started a run last night for GPT-3
11:05 the chips are cooking right now at a major fab overseas
12:58 gated convolutional attention, and the layer they haven't cracked
14:03 the Hardware Lottery, and why he wants people gambling again
15:20 what happens if inference gets 1,000x more efficient
16:59 why you can do this research without a basement full of GPUs
Show more
The masculine urge to Speedrun building the Dyson Swarm
If you are reading this, you are so early.
Start digging in
Extropic founder @beffjezos predicts 1000X more efficient AI inference could have a multi-trillion-dollar impact on the economy:
"It shows it's possible to create arbitrarily scalable models on a new hardware substrate. We released our first results about a year ago, and it was MNIST, which is the very early days. It's the '80s of machine learning."
"Today we have models that are GPT-2 level that can run on our hardware. Algorithmic progress is blazing fast."
"If we can get to 1000x greater energy efficiency for inference sooner rather than later, that changes the whole world. That has a multi-trillion dollar impact on the economy."
"Even if you're new to machine learning, you can vibe code this. You can use auto research. This is a whole new territory that's barely explored. It's just been a couple of our scientists."
@extropic
Show more
Crazy stuff. This caught my eye in the technical report...
Disaggregated inference between Thermodynamic Sampling Units (TSUs) and XPUs / FPGAs.
As I understand it, the TSU is extremely energy efficient at the sparse neural computations this new sparse transformer they've built does so running all of those on the TSU and everything else on an XPU or FPGA is much more energy efficient. It's also faster.
Show more
Introducing Z1T:
Our first family of transformer-like models
made for sparse probabilistic hardware like Z1
Achieving up to 140x energy efficiency gains over GPUs
and revealing a new scaling law for sparse transformers
Read the blog:
Show more
And for my next trick, I'll demonstrate algorithmic progress going from MNIST to GPT-2 in a year for a whole new hardware paradigm 🪄🎩
The only path forward is AI Safety through multipolar adversarial capabilities equilibrium
