註冊並分享邀請連結,可獲得影片播放與邀請獎勵。

Jukan
@jukan05
加入 September 2024
453 正在關注    226.6K 粉絲
BofA’s Vivek Arya made an excellent point, which I’d like to share. Is open-source AI bearish for memory? Closed models amortize global demand across shared HBM pools concentrated in a handful of data centers, whereas open models create a new memory footprint with every deployment. If 10,000 companies self-host the same open model, the model weights must be replicated across 10,000 separate HBM pools, with each deployment also requiring its own KV cache. As 128K–1M token contexts become commonplace in 2026, the KV cache alone can exceed 40GB per active session. Low-cost Chinese APIs drive greater inference demand, broader enterprise self-hosting, and more memory sockets worldwide. In short, closed models concentrate memory demand, while open models multiply it.
顯示更多
0
35
807
84
轉發到社區