Zijing Wu
@zijing_wu
Technology and all things interesting @FT in HK. Returning to journalism (ex BBG) after a decade as investor/entrepreneur. Still an idealist. Opinions mine.
加入 May 2009
563 正在關注 11.8K 粉絲
Was told it takes only about 30K GPUs to pre train a mega model. Inference compute is the real monster. But then most likely they’ll just serve distilled smaller models.
Chinese AI labs seem confident they have the compute needed to pre-train 5-10 trillion parameter models.
Someway, somehow compute does not seem to be such a major bottleneck, at least when it comes to reaching these levels of model size.
顯示更多
