Kaiyue Wen
@wen_kaiyue
A continuous learner
加入 June 2023
1.1K 正在关注 2K 粉丝
A hidden detail in the recently released Muse Glimmer model: its per-matrix weight RMS norm is pinned almost exactly at ~6e-3!
If you’re curious why fixing weight RMS like this can work, feel free to checkout Hyperball:
显示更多
