Kaiyue Wen
@wen_kaiyue
A continuous learner
가입 June 2023
1.1K 팔로잉 중 2K 팬
A hidden detail in the recently released Muse Glimmer model: its per-matrix weight RMS norm is pinned almost exactly at ~6e-3!
If you’re curious why fixing weight RMS like this can work, feel free to checkout Hyperball:
더 보기
