I bet this distributed inference will look pretty simple — everything on the edge. That’s Apple’s play and the M silicon does great.
Combine with OS models and you’re only six months out of date (which is just fine for a ton of people) at zero marginal cost
I am pretty convinced that there will be a path for distributed AI inference and this will create a novel way for folks who may not otherwise be in the AI economy to make money from it.