Larry Dial(@classiclarryd):Building momentum at Marin! Upgrading from Dense -> 129B parameter MoEs -> architecture improvements -> optimizer improvements gives our pretraining recipe an estimated 6x cumulative learning speedup, accounting for MFU. Includes community contributions.

Larry Dial

@classiclarryd

Technical Staff at Open Athena, working on Marin

Joined May 2024

48 Following 2.2K Followers

Larry Dial@classiclarryd

2026.06.03 16:26

Building momentum at Marin! Upgrading from Dense -> 129B parameter MoEs -> architecture improvements -> optimizer improvements gives our pretraining recipe an estimated 6x cumulative learning speedup, accounting for MFU. Includes community contributions.