Zijing Wu
@zijing_wu
Technology and all things interesting @FT in HK. Returning to journalism (ex BBG) after a decade as investor/entrepreneur. Still an idealist. Opinions mine.
Joined May 2009
563 Following 11.8K Followers
Was told it takes only about 30K GPUs to pre train a mega model. Inference compute is the real monster. But then most likely they’ll just serve distilled smaller models.
Chinese AI labs seem confident they have the compute needed to pre-train 5-10 trillion parameter models.
Someway, somehow compute does not seem to be such a major bottleneck, at least when it comes to reaching these levels of model size.
Show more
