Lisan al Gaib(@scaling01):it's called the forbidden axis for a reason well, except it's no longer forbidden. OpenAI opened the floodgates and will release their second looped language model besides GPT-6 Astra on DevDay

2026.09.24 02:40

it's called the forbidden axis for a reason well, except it's no longer forbidden. OpenAI opened the floodgates and will release their second looped language model besides GPT-6 Astra on DevDay

Samip@industriaalist

2026.09.22 20:58

We think computational depth is the missing scaling axis, i.e. we should be doing a lot more deep learning! Every other axis has been scaled by OOMs over the past few years (params, data, sparsity, test-time reasoning), but depth has been stuck at ~100 layers since GPT-3. We've found that LLMs are both *severely* depth-bottlenecked and bad at using the depth they have, and that architectural interventions that lift this bottleneck efficiently lead to gains that increase with compute. w/ @akshayvegesna