Register and share your invite link to earn from video plays and referrals.

wh
@nrehiew_
eng primarily, ml mostly, research previously
Joined October 2023
108 Following    19K Followers
During the run, we wanted a clearer picture of model progress and monitored both partial credit and the gain on only the fair tasks of DeepSWE. We often saw pass@1 gains trailing behind pass@8. And it turns out that the pass@8 gains were the model simply lucking into a correct solution on the unfair tasks. We found that both these approaches were more representative of capabilities beyond binary reward.
Show more