登録して招待リンクを共有すると、動画再生報酬と紹介報酬を獲得できます。

dylan ツ
@demian_ai
growth @nebiustf @nebiusai // ex @Scaleway // from silicon to token, inference and anything in between. Views are my own - not financial advice
参加 January 2022
2.7K フォロー中    30.7K ファン
the TPU is still the star of google's ai story, but the chip sitting next to it might decide how fast your answers come back. $MRVL investor day yesterday spelled out its google deal: custom chips that plug into the TPU world, inference accelerators included, worth up to $120B through 2033 if Google hits its purchase targets. a second inference only chip reportedly paired two to one with tpus (treat the ratio as rumor) it makes sense once you split a reply into its two jobs: 1. reading your prompt is a math job, and a TPU or GPU eats it for breakfast 2. Writing the answer word by word is a memory job, because for every token the chip has to pull the model and your whole conversation back out of memory. Etched's CEO @UbertiGavin said it plainly on invest like the best: decode is all a memory game. on blackwell, he said, one hop between gpus can take about 4,000 nanoseconds, which is why eight gpus don't make you eight times faster. So the hardware is splitting along that line. One chip for thinking about the prompt, another built to talk back fast. That puts the pressure on memory bandwidth across the whole cluster and on the wires between chips. No surprise that interconnect is about $37.5B of the growth $MRVL laid out
もっと見る