Yu Zhang ๐๐
@yzhang_cs
@Kimi_Moonshot; working on efficient methods for LLMs; disciple of parallel programming; INTP; ๐
925 Following 6.8K Followers
> for instance kimi K3 created its own experiment API for generating optimizer variants, loss comparisons, Newton Schulz tuning etc.
awesome๐ถ๏ธ๐ถ๏ธ๐ถ๏ธ, speedrun was also what @nathancgy4 and I had in mind for a hill climbing showcase โ didn't expect K3 to be this good at it.
Kudos!
Show more
Quick sync alongside the K3 report: FLA v0.5.2 is out ๐ข
- KDA fused kernels w/ FlashKDA backend
- KDA CP support
- Fused attnres
Enjoy it! cc @SonglinYang4 @uniartisan
