註冊並分享邀請連結,可獲得影片播放與邀請獎勵。

wh
@nrehiew_
eng primarily, ml mostly, research previously
加入 October 2023
108 正在關注    19K 粉絲
During the run, we wanted a clearer picture of model progress and monitored both partial credit and the gain on only the fair tasks of DeepSWE. We often saw pass@1 gains trailing behind pass@8. And it turns out that the pass@8 gains were the model simply lucking into a correct solution on the unfair tasks. We found that both these approaches were more representative of capabilities beyond binary reward.
顯示更多