가입 후 초대 링크를 공유하면 동영상 재생 및 초대 보상을 받을 수 있습니다.

Binkg
@dummerspast39
anti ai lobby china shill big hater
가입 August 2024
289 팔로잉 중    18.9K 팬
Baidu killed the MTP module everyone copied from DeepSeek ERNIE 5.5 Flash just runs its own last layer a second time to guess the token after next. They call it ECHO. Trained on only 1 extra token and it still drafts several ahead for speculative decoding, w/ higher acceptance than an MTP module trained the same way With a normal MTP module the future token loss mostly trains the bolted on module. ECHO puts that loss on the real model so the backbone itself learns to look ahead, and Baidu says reasoning and code got clearly better. The leftover ECHO layer is like 10 times smaller than the MTP module tencent bolted on Hy3, a model the same size
더 보기