가입 후 초대 링크를 공유하면 동영상 재생 및 초대 보상을 받을 수 있습니다.

wh
@nrehiew_
eng primarily, ml mostly, research previously
가입 October 2023
108 팔로잉 중    19K 팬
We also had to do a bunch of tricks to keep training efficiency high: 1) You can't do weight sync the standard way in such a setup. So we do sharded HTTPs, which maximises the egress bandwidth while keeping as little overhead as possible. It turns out that there isn't much point doing delta weight sync for LoRA since the overhead of decoding the weights is not worth it given LoRA is already small in size Also, you could actually end up being CPU bound here on weight sync. If your inference replicas are busy, you dont want to add more non prefill/decode workload dealing with weight sync. Especially the case with LoRAs where the whole benefit is that you can still continue inference on rollouts from past adapters easily
더 보기