註冊並分享邀請連結,可獲得影片播放與邀請獎勵。

Binkg
@dummerspast39
anti ai lobby china shill big hater
加入 August 2024
289 正在關注    18.9K 粉絲
Only 6 of ERNIE 5.5 Flash Base's 43 layers see every token individually >the other 37 squash every 128 tokens into 1 cached entry, plus a short window of recent tokens >those cheap layers get wide 512-dim heads, the 6 full-context layers get 256-dim >the 6 full-context layers use sparse attention, only over the top-k scored positions >still beats DeepSeek V4 Flash Base on 4 of 6 long-context benchmarks Baidu built the KV cache on a diet
顯示更多