註冊並分享邀請連結,可獲得影片播放與邀請獎勵。

elie
@eliebakouch
research @PrimeIntellect (prev: @huggingface) anon feedback:
加入 January 2024
4.8K 正在關注    26.8K 粉絲
huge congrats to reflection on the release, very excited that they will release the weight under apache 2.0!! i'm quite confused by what seems to be a very low MFU for pre training. if you assume ~80% goodput for 4 week (which is already low) you get around ~12% BF16 MFU arch seems to be global/SWA interleaved with a 3:1 ratio, with slightly lower parameter sparsity than other oss models (~5%). their training also seems to be quite stable and they get a better base on held out code ppl than dsv4!
顯示更多
0
5
180
5
轉發到社區