TOKENMAXXING: @vllm_project and @inferact achieve 1.3 BILLION tokens per $ of TCO (hyperscaler volume pricing) via disaggregated serving on AgentX. This result is 4x better than the B300 aggregated configuration.

True Ventures @madelineminshew says the era of tokenmaxxing is over: spending $10K a day used to be cool. Now it's "what are you doing?" "Cost is a really big thing. For a long time, token maxing was super hot, and it was cool to say, I'm spending 10K a day. And now it's like, what are you doing?" "Sometimes it's definitely worthwhile to have that spend. If you are a small engineering team and you're thinking about your token cost as an employee cost, or if you are tackling a project that needs that, great." "But oftentimes, the hardest work that needs to be done once to prove something or to build something, you can then switch to an open source model after that." "Uber's CTO has spent a lot of time talking about how they're handling this: you do some of the hardest work on the frontier models, and after that, there's a real big push across their entire portfolio right now to find good open source alternatives." @trueventures @tonysphere