註冊並分享邀請連結,可獲得影片播放與邀請獎勵。

Silicon Data
@silicondata
Revolutionize compute markets through unparalleled data transparency, financial product innovation and market intelligence
加入 May 2024
27 正在關注    26.1K 粉絲
In this week’s Reference Price newsletter we flagged an interesting “generational slope” in on-demand GPU rental rates. Over the past week the increases were ordered by chip generation: B300 +8.6% → B200 +4.7% → H200 +4.2% → H100 +2.5% → A100 +2.0%. To the extent there is a compute shortage, it appears concentrated at the newest chips. Long-context inference is increasingly memory-bound, which favors the chips with the most high-bandwidth memory — and that memory remains tight. The question is whether rents are starting to separate by capability rather than rising uniformly. Worth watching closely to see if the slope persists. Over the weekend @drwconvexity and @GavinSBaker discussed the “crunch spread” (relative moves in GPU rental rates vs. usage-weighted token prices). @deedydas also noted that the most advanced GPUs continue to deliver rising token throughput efficiency, which raises a related question: how much of the rental premium is justified by throughput gains versus pure scarcity of the newest hardware?
顯示更多
Another day, another round of on-demand GPU rental indices surging across the board and reaching ATHs. Even the A100 index, which recently saw some softness, seems to be inflecting a little.
顯示更多