登録して招待リンクを共有すると、動画再生報酬と紹介報酬を獲得できます。

Binkg
@dummerspast39
anti ai lobby china shill big hater
参加 August 2024
289 フォロー中    18.9K ファン
Baidu killed the MTP module everyone copied from DeepSeek ERNIE 5.5 Flash just runs its own last layer a second time to guess the token after next. They call it ECHO. Trained on only 1 extra token and it still drafts several ahead for speculative decoding, w/ higher acceptance than an MTP module trained the same way With a normal MTP module the future token loss mostly trains the bolted on module. ECHO puts that loss on the real model so the backbone itself learns to look ahead, and Baidu says reasoning and code got clearly better. The leftover ECHO layer is like 10 times smaller than the MTP module tencent bolted on Hy3, a model the same size
もっと見る