Our serverless APIs for open weights models have become exquisite machinery (that we plan to write more about soon!) One effect is that @togethercompute now one of the largest originators of open tokens in the US, with a wild week over week growth curve that reflects the pace of OSS adoption. Will serve 1.8T tokens today on @OpenRouter alone. You can get started with $5 —

Nice stats from @OpenRouter that illustrate how @togethercompute delivers solid and scaled performance for agentic workloads. We are serving GLM 5.3 and GLM 5.3 Flash @ top decile of TPS, latency, cache rate, and doing it at large volumes, 23% and 30% of all OpenRouter traffic, and OpenRouter is a fraction of our overall API traffic. Running scaled inference for agentic applications is a lot more than maxxing a single metric ... you need to optimize on all dimensions and doing so at scale with reliability.

My first byline for @CNBC, co-reported with @KatieTarasov - how are legacy data center operators like @Equinix fitting into the AI boom? It's all in the inference (and a partnership with @nvidia and @togethercompute).

At Equinix Horizon, Adaire Fox-Martin asked Jensen Huang why inference needs to sit close to your data, not just more compute. Today that became Equinix® Inference Exchange, built with @nvidia + @togethercompute.

AI inference has a location problem: where should workloads run, and how should they connect? Equinix® Inference Exchange brings together #Equinix#, @nvidia and @togethercompute to help enterprises move from AI experimentation into production. Learn more:

We are partnering with @Equinix and @nvidia to build a global inference fabric for inference. This partnership brings inference (literally) close to enterprise data and applications through our new inference edges colocated in Equinix's distributed, enterprise-class, low-latency datacenters.

We executed a landmark partnership with @HUMAIN today to offer open source models on 250 mw (-100K+ chips) of capacity over the next 12 months. The demand for open and custom AI tokens is vertical and this partnership represents one of the largest so far focused on delivery OSS tokens globally.

We recently introduced support in @togethercompute endpoints for seamless A/B experiments and rollouts. Our customers are using this to rollout new weights safely and rapidly for small to large deployments.

Fantastic initiative by @nvidia. “Circular financing” is thrown around as a pseudo intellectual gotcha, while everyone in the AI infrastructure business is witnessing demand curves that are so rapidly outpacing supply that price escalations and demand destruction are now accepted as default strategies for what is arguably the most productive and promising technology of our time. So far it has required hyperscalers to underwrite AI infrastructure at scale, but demand is larger than what hyperscaler balance sheets can securitize. It also skews the market to a small number of providers, gives them unbounded pricing leverage and defaults to their specific product strategies. With NVIDIA’s underwriting program, non-investment grade innovative startups and enterprises will be able to participate in building tomorrow’s AI infrastructure. This will result in better and more diverse products, faster AI transformation of the American industry, and a healthier and less concentrated AI infrastructure ecosystem.

This is huge news for India, Chennai, and L&T. Larsen & Toubro (L&T) has secured a major deal to build India’s largest NVIDIA B300 AI Factory. In partnership with US based AI company Together AI, L&T will set up a massive AI computing facility powered by 10,000 NVIDIA B300 GPUs. This will be India’s largest single cluster AI infrastructure of its kind. The facility will come up at Vyoma, AI’s data centre campus in Chennai and will power Together AI’s cloud platform for large scale AI training and inference. The Chennai site is built for big expansion, with Phase 1 designed for up to 250 MW of power capacity. This move significantly strengthens India’s AI infrastructure and takes the country a step closer to becoming a global hub for next generation AI. Congrats to everyone who made this happen.

Some people are interpreting this as a way for OSS models to bypass encryption of closed model APIs. If that is what you are taking away from this, please understand that the title is a massive overstatement in this context. The point of encryption here is to keep the inference protocol stateless; I assume labs are fully aware of some side channel implications and do not care about preventing replay (why would they?) Encrypted traces offer a stateless inference protocol so inference backend can be distributed across hundreds or thousands of data centers without the need for coordinating sessions. If your side channel is to ask the LLM "true or false" questions for the universe of words that might be contained in a trace, you'd go broke decoding a single trace. So you'd have to assume there's some super special magic trace that lived within some encrypted blob that's worth attacking in this way. Overall definitely an interesting limitation of the encrypted vs stored traces, but you aren't going to be able to steal traces this way to train a competing model. That would cost more than building the model with the standard training arsenal.