가입 후 초대 링크를 공유하면 동영상 재생 및 초대 보상을 받을 수 있습니다.

Avi
@avikrishna
overdosing on information
가입 August 2018
1.5K 팔로잉 중    1.2K 팬
re: Fast followers in AI In distillation, you have a student model sampling traces from a teacher model. But in certain cases, the student may receive such high quality traces that it surpasses the teacher model, despite the teacher being much larger. Sonnet 3.5 is a real example of this: despite being a smaller and cheaper model than Opus, it performed noticeably better! So when people bring up the water skier-boat analogy to say that the water skier (the student) always lags the boat (the teacher), they are wrong. The student can always surpass the teacher. This is also why the rhetoric of a certain senator-elect is confusing: he argues that we can and should pause AI because China is nothing but a fast follower. So while this infantilization is wrong on its face given how many Chinese researchers are at American labs, it is also, on a technical level, incorrect.
더 보기