A year ago the consensus was everyone would be using one or two frontier models, running entirely on NVIDIA GPUs. The reality now is we have hundreds of models each with different cost/speed/intelligence tradeoffs for different tasks, running on dozens of different local / data center chips (even older hardware). Ironically the frontier models that were trained on NVIDIA GPUs have made it faster and easier to port kernels from CUDA to other hardware targets (AMD, Apple Silicon, Intel, ...). There are people with no experience writing kernels using relatively cheap AI agents to implement efficient kernels for Apple Silicon, synthesizing all the tricks from existing CUDA implementations that were meticulously hand written by experts. Funnily enough, since the performance of these kernels can be quickly and objectively evaluated by an agent by actually running the kernel on the hardware, it's an unreasonably tractable task for agents. They can continuously improve them in a fairly simple autoresearch loop. Kernel interoperability is solved ("The unreasonable effectiveness of AI agents"). The next continuation of this trend is to have millions of small, specialized models for each use case, tuned for each user based on how they use the models / their SLAs (e.g. maybe you're fine running a job overnight so it can run on cheaper hardware). They will run efficiently on any hardware, so then it's a choice of which hardware is the best for that specific thing. For this we need better infrastructure to evaluate models and map out the tradeoff space so we can give the optimal point in the tradeoff space of Model x Quant x Harness x Inference Engine x Config x Hardware.