Larry Dial(@classiclarryd):Building momentum at Marin! Upgrading from Dense -> 129B parameter MoEs -> architecture improvements -> optimizer improvements gives our pretraining recipe an estimated 6x cumulative learning speedup, accounting for MFU. Includes community contributions.

Larry Dial

@classiclarryd

Technical Staff at Open Athena, working on Marin

加入 May 2024

48 正在關注 2.2K 粉絲

Larry Dial@classiclarryd

2026.06.03 16:26

Building momentum at Marin! Upgrading from Dense -> 129B parameter MoEs -> architecture improvements -> optimizer improvements gives our pretraining recipe an estimated 6x cumulative learning speedup, accounting for MFU. Includes community contributions.