Adoption ≠ Contribution: Some Thoughts on Open-Source Inference Engines A few thoughts on how open-source LLM inference engines get adopted. Users usually fall into a few groups: model providers, neo labs, inference clouds, and neo clouds. There’s some overlap between them. For inference clouds and neo clouds, adoption is usually about getting the engine running. Fixes and optimizations often stay in private forks because there isn’t much incentive to push them upstream. For model providers and neo labs, moving fast is usually more important. Some keep developing on an old internal fork and gradually drift further away from upstream. Others rebase on main only occasionally. Either way, their internal fixes and optimizations don’t necessarily make it back to the open-source project. That’s why a lot of the contributions to open-source inference engines still come from the engine owner teams and vendors like NVIDIA and AMD. Also, “we run it at scale” doesn’t always tell you much about the upstream project. The internal fork can be quite different from what everyone else is using. Without a real feedback loop from production back to upstream, adoption alone doesn’t help the ecosystem that much. The adoption we value most is when people actually co-evolve with the project: run it in production, report what breaks, sync with upstream, and contribute fixes back. This is also why the engine itself is hard to turn into a product, and even harder to turn into a business. Whether you sell GPU hours, tokens, or RL services, you’re ultimately selling a service built on the same GPUs. Adding a service layer can improve the margin, but today that layer is still pretty thin, and many of the capabilities are relatively easy to replicate. The interesting part is what happens when that service layer gets deeper—better products, better user experience, and capabilities that are actually hard to reproduce. That’s where the economics start to make more sense.