it's worth noting that due to rising costs of pretraining, the frontier isn't *that* accessible. training a 5T-A100B model on 50T tokens on GB300s costs like $10 million for the hero run. double it for RL. there's only a few thousand companies in the world that could justify it
もっと見る
