18x improvement in intelligence per joule in 16 months is a insane @amasad.
The interesting part is a lot of this improvement has been attributed to model efficiencies.
A good chunk of it will be hardware, given that the chip you run the model on sets the floor for joules per token before you write any optimization. Look at May 2024
IMO, the next order of magnitude on this chart over the coming years will come from purpose built inference hardware, not just better kernels.
Great research from @Avanika15 and @JonSaadFalcon.
Show more
