David Hall
@dlwh
Member of Technical Staff @ Open Athena. Creator of Levanter and Marin. Previously Research Engineering @StanfordCRFM, co-founder at Semantic Machines ⟶ MSFT.
1.3K Following 3.6K Followers
Also, the post has a math duel from a Renaissance beef over cubic equations.
Anyway, here’s our latest architecture and data mix from a run we kicked off over the weekend. Looking like a pretty good run so far. Entire recipe is public! Wandb:
Show more
Frontier AI models are built from thousands of small decisions: data sourcing, filtering, mixtures, curricula, scaling experiments, optimizer choices, kernels, evals, failed runs, and protocols for deciding what gets scaled.
This is process knowledge.
Show more
