註冊並分享邀請連結,可獲得影片播放與邀請獎勵。

ellington
@not_ellington
Computers
加入 October 2016
190 正在關注    4.5K 粉絲
Prediction ML silicon started off unified (single GPUs/TPUs) Then grew into larger sharded clusters (NVL72s) And is now beginning to grow into more heterogeneous, disaggregated structures. Inference providers are realizing they can get better tok/watt using GPUs for prefill/FFNs and WSE/Raptors for decode/attention layers and CPUs for agent orchestration. I think that this branching will continue as we get independent chips for projections, FFNs, attention, normalizations, tool-use, etc. A dataflow-based rack entirely designed around the interconnect and data-locality to minimize power. I think we will look back in 5 years on the idea of using the same cores for both FFNs and attention layers as absurd. The science of silicon is extremely advanced, but the engineering putting it to use is still in its infancy. Much of this is because we were able to lean on Moores law for so long we could afford to not worry so much about our hardware designs. Now this naivety is beginning to show itself.
顯示更多
0
9
127
13
轉發到社區