Larry Dial(@classiclarryd):Building momentum at Marin! Upgrading from Dense -> 129B parameter MoEs -> architecture improvements -> optimizer improvements gives our pretraining recipe an estimated 6x cumulative learning speedup, accounting for MFU. Includes community contributions.

Larry Dial

@classiclarryd

Technical Staff at Open Athena, working on Marin

가입 May 2024

48 팔로잉 중 2.2K 팬

Larry Dial@classiclarryd

2026.06.03 16:26

Building momentum at Marin! Upgrading from Dense -> 129B parameter MoEs -> architecture improvements -> optimizer improvements gives our pretraining recipe an estimated 6x cumulative learning speedup, accounting for MFU. Includes community contributions.