Kaiyue Wen
@wen_kaiyue
A continuous learner
Joined June 2023
1.1K Following 2K Followers
A hidden detail in the recently released Muse Glimmer model: its per-matrix weight RMS norm is pinned almost exactly at ~6e-3!
If you’re curious why fixing weight RMS like this can work, feel free to checkout Hyperball:
Show more
