가입 후 초대 링크를 공유하면 동영상 재생 및 초대 보상을 받을 수 있습니다.

elie
@eliebakouch
research @PrimeIntellect (prev: @huggingface) anon feedback:
가입 January 2024
4.8K 팔로잉 중    26.7K 팬
huge congrats to reflection on the release, very excited that they will release the weight under apache 2.0!! i'm quite confused by what seems to be a very low MFU for pre training. if you assume ~80% goodput for 4 week (which is already low) you get around ~12% BF16 MFU arch seems to be global/SWA interleaved with a 3:1 ratio, with slightly lower parameter sparsity than other oss models (~5%). their training also seems to be quite stable and they get a better base on held out code ppl than dsv4!
더 보기