登録して招待リンクを共有すると、動画再生報酬と紹介報酬を獲得できます。

Avi
@avikrishna
overdosing on information
参加 August 2018
1.5K フォロー中    1.2K ファン
re: Fast followers in AI In distillation, you have a student model sampling traces from a teacher model. But in certain cases, the student may receive such high quality traces that it surpasses the teacher model, despite the teacher being much larger. Sonnet 3.5 is a real example of this: despite being a smaller and cheaper model than Opus, it performed noticeably better! So when people bring up the water skier-boat analogy to say that the water skier (the student) always lags the boat (the teacher), they are wrong. The student can always surpass the teacher. This is also why the rhetoric of a certain senator-elect is confusing: he argues that we can and should pause AI because China is nothing but a fast follower. So while this infantilization is wrong on its face given how many Chinese researchers are at American labs, it is also, on a technical level, incorrect.
もっと見る