Subquadratic
@subquadratic
AI lab leading the subquadratic LLM revolution.
1.1K Following 20K Followers
New Podcast Alert.
Today on #GAEATalks#, @alex_whedon and Graeme Scott go deep on the macroeconomics of AI, why an efficiency-focused architecture is not a nice-to-have but an inevitability, and the two fundamental flaws of Transformer based models that are quietly capping what AI can do.
Click here for the full episode.
Show more
"The more data you send to the model the less efficient it gets.
This is a major problem that leads to deficiencies in the model's ability to reason over large amounts of data, it limits the amount of training that we can do on a fixed amount of compute, and it creates skyrocketing costs which we're seeing very much in the market today.
As someone who has been working in the space for 10 years, this was a very important problem to solve.”
@alex_whedon & @Justin_SubQ from @subquadratic had a great conversation with @sophiadew on @MTSlive.
Check out this clip from their interview:
Show more
