가입 후 초대 링크를 공유하면 동영상 재생 및 초대 보상을 받을 수 있습니다.

wh
@nrehiew_
eng primarily, ml mostly, research previously
가입 October 2023
108 팔로잉 중    19K 팬
2) A big part of what we want to do is realistic experiments so rollouts often hit a model's max context. For example with Qwen 3.8 27B, it is a dense model and doing long-context RL on it requires quite abit of work. At this context length, attention completely dominates your profile. You don't want to recompute attention twice but you also cant cache the whole attention layer or you just run out of memory even on a B300. The solution we came up with is to only cache the FA4 output and LSE, recomputing everything else. This maximises memory usage (while still not OOMing) and speeds up step time by a non-trivial amount.
더 보기