Leo the Horseman (prediction arc)
@LeotheHorseman
Hyperreality Capitalist | Chief Janitor@████████
参加 October 2022
4.5K フォロー中 16.4K ファン
牛逼
We live in a multimodal world. We see, talk, act, and dream.
Yet most LLMs still start with language pretraining. Why not train them natively with multimodal I/O from scratch?
Because it’s SUPER HARD, adding modalities triggers training instability, design complexity, and often modal competition
So what’s the path forward?
Introducing: Towards Physics of Multimodal Pretraining (
We unpack the underlying mechanics of multimodal pretraining across 4 aspects: Knowledge Flow, Modality Synergy, Early Unification, and Recipe.
もっと見る
