Sonnet Xu
@sonnet_xu
cs @ stanford || reliability + benchmarking, compute + llms
89 Following 1.3K Followers
inference is becoming a market-making business
upstream, GPU capacity is bought in chunks: financed, contracted, take-or-pay, multi-year
downstream, tokens are bought on demand
warehousing the mismatch between the two means forecasting demand, absorbing idle capacity, and repricing compute into tokens
neoclouds are moving down into inference and inference providers are moving up into GPUs whoever can make a market between fixed capacity and variable demand takes the spread
Show more
