Kaiyue Wen
@wen_kaiyue
A continuous learner
参加 June 2023
1.1K フォロー中 2K ファン
A hidden detail in the recently released Muse Glimmer model: its per-matrix weight RMS norm is pinned almost exactly at ~6e-3!
If you’re curious why fixing weight RMS like this can work, feel free to checkout Hyperball:
もっと見る
