Dr. Mike Israetel
@misraetel
-PhD in Sport Physiology and former Prof. of Nutrition and Sport Science
-Cofounder of Renaissance Periodization
-BJJ Black Belt, Competitive Bodybuilder
Joined August 2015
327 Following 38.1K Followers
Thinking broader AND deeper. Pondering. This is a big deal.
GPT-6 Astra reportedly gets deeper without getting bigger. The labs already know what that means for scaling.
"It's basically confirmed that GPT-6 Astra uses loop transformers, which means that instead of adding parameter count, it goes through the layers more than once. You add compute depth, but you don't increase the size of the model."
"The labs are probably the people that are best positioned to say which way models are scaling. I think that's a tell that they're not seeing parameter sizes scaling as aggressively in their roadmaps, in what they find in their research."
Show more
