注册并分享邀请链接,可获得视频播放与邀请奖励。

Avi
@avikrishna
overdosing on information
加入 August 2018
1.5K 正在关注    1.2K 粉丝
re: Fast followers in AI In distillation, you have a student model sampling traces from a teacher model. But in certain cases, the student may receive such high quality traces that it surpasses the teacher model, despite the teacher being much larger. Sonnet 3.5 is a real example of this: despite being a smaller and cheaper model than Opus, it performed noticeably better! So when people bring up the water skier-boat analogy to say that the water skier (the student) always lags the boat (the teacher), they are wrong. The student can always surpass the teacher. This is also why the rhetoric of a certain senator-elect is confusing: he argues that we can and should pause AI because China is nothing but a fast follower. So while this infantilization is wrong on its face given how many Chinese researchers are at American labs, it is also, on a technical level, incorrect.
显示更多