Matej Sirovatka
@m_sirovatka
head of hr @ prime intellect | int64 upcaster
参加 August 2021
538 フォロー中 3.9K ファン
you can do blazingly fast weight transfer in prime-rl now, spent quite a while on this (imo quite cool) solution and finally happy with the performance, give it a try
Our RL stack now supports NIXL weight transfer, reducing trainer-to-inference transfer time 9x compared with NCCL: from 86 seconds down to single-digit seconds for an 800B-parameter model, and even <4 seconds in our experiments.
For prime-rl users, this means over 25% more throughput end-to-end compared with our previous speed. It also clears the way for fault-tolerant, elastic inference scaling that NCCL's rigid process groups made difficult.
もっと見る
