Inferact now serves more than 7 trillion tokens a day. That traffic runs on the Inferact Platform, which serves frontier models with optimized vLLM across heterogeneous hardware and diverse workloads. For context, OpenRouter averaged 19 trillion tokens a day of open-weight traffic over the past 30 days, across all of its providers. The platform improves through a data flywheel. Production traffic shows us where to improve vLLM, partnerships with accelerators like NVIDIA, AMD, and Google bring those improvements to new hardware, and partnerships with model labs like MiniMax and Kimi bring them to new models from launch day. Next, we’re automating that flywheel with agents that find and validate optimizations on their own, so the platform improves with every token it serves.

Today, @Kimi_Moonshot launched Kimi K3.1 Pro, and we are announcing an expanded partnership with Moonshot AI to bring the Kimi model series to production with vLLM. Kimi K3.1 Pro is available on the Inferact Platform from launch. We held it to the same standard as Kimi K3, which we brought onto the platform after completing final qualification and Kimi Vendor Verification. Inferact also partners with Moonshot AI on @vllm_project support for each Kimi release. For Kimi K3.1 Pro, developers can serve the model in vLLM today. Our work with Moonshot AI on each model starts before release. We tune and validate the model serving path together, then ship improvements to the community, such as a speculative decoding model that raises generation throughput. What we learn serving Kimi in production is upstreamed to vLLM. Thank you to the Moonshot AI team for the partnership. Read the full announcement:

Our teammate Ziming Huang joined Ascend's inference meetup on October 22 to share how we take @Kimi_Moonshot's Kimi K3 from day-0 support to production serving. Early model access, architecture understanding, and the runtime and kernel work behind KDA, MLA and LatentMoE—the engineering behind making a model ready to serve. Thanks to Huawei and the Ascend community for having us!

Today, we are announcing a strategic partnership with @MiniMax_AI to advance frontier production inference for open-source models. The Inferact Platform has served MiniMax M3.1 Flash in production since the day it launched. Requests through the MiniMax platform are routed to our stack alongside MiniMax’s own capacity, after we met MiniMax’s vendor verification requirements for accuracy and reliability. The Inferact Platform serves this agentic workload with guaranteed SLAs and auto-tunes the endpoint against live traffic, while MiniMax keeps control of how its model is served. Inferact also leads day-0 @vllm_project support for each MiniMax release, so the model serves at production quality the day the weights ship. The engineering from both efforts is upstreamed to vLLM. Thanks to the MiniMax team for the partnership. Read the full announcement:

We’re heading to #PyTorchCon#! 🎤 Catch our talks and meet the team at Inferact booth P1. ☕ Join us, Anyscale & Nscale at the Inference & RL Coffee Club, Oct 20–21, noon–5 PM PT: Looking forward to seeing you there! 🙌

Inferact is an API launch partner for MiniMax M3.1 Flash! 🎉 We're working with @MiniMax_AI on production inference for its latest model, built for multi-step automation, real-codebase engineering and visual reasoning. M3.1 Flash scored 29.6 on Agents’ Last Exam (ALE), alongside 72.5% on AutomationBench-AA and 73.67% on DeepSWE—capabilities that matter when agents need to plan work, use tools and carry tasks through. Our contribution is the serving stack: bringing our vLLM engineering and the Inferact Platform to the collaboration. Thanks to the MiniMax team for the partnership. Explore the model and API integration:

Today, we are announcing an engineering collaboration with @AMD to strengthen production agentic inference on AMD Instinct GPUs in @vllm_project. AMD brings the Instinct platform, ROCm engineering, and the AITER kernel library, while Inferact and the vLLM community bring model integration, serving features, and day-0 enablement. Together, we are focused on agentic workloads with long-running sessions, long prefixes, and tool calling, and each configuration will ship as a public per-model serving recipe. This work is already running in production. It shipped with this summer’s major open-model releases, and Inferact uses it on AMD Instinct clusters for our Inferact Platform customers. Everything this collaboration produces lands in open source. Read the full announcement:

Our new website is live! Along with a new visual identity for Inferact. Inferact gives AI teams production inference they control, run optimized inference wherever your compute lives. Come check it out

Thank you @a16z for telling the Inferact story. Four years in, inference is still far from solved. Models keep getting bigger, workloads keep getting more varied, and the hardware landscape keeps expanding. Inferact exists to build the future of inference with the vLLM community. We’re hiring across all teams in SF and Singapore:

Thanks for the shoutout @SemiAnalysis_! Our team at @inferact has been optimizing @vllm_project on GB300, and those gains translate directly into better TCO. More optimizations are coming soon. Current results on InferenceX:

Great collaborating with @deepseek_ai, @nvidia, and @SemiAnalysis_ to build this with the @vllm_project community. Dropping TTFT nearly 70% at ~100K throughput reflects the joint work across models, custom kernels, and engine serving. vLLM is the open source engine; Inferact optimizes it and brings enterprise inference to production.

Come join us at vLLM Meetup Toronto! Our Co-founder and Lead vLLM Maintainer @rogerw0108 will be speaking on where @vllm_project is heading next, alongside engineers from Cohere and NVIDIA. The event is free, but spots are limited. Register via the link in the thread.

Runtime is this week at The Midway in San Francisco. Thanks to our partners for helping us put it together: @nvidia, @pydantic, @radixark, @sgl_project, @temporalio, and @inferact, @vllm_project 💚 Join us to learn from the engineers behind these tools.

Our first TPU megakernel for Kimi K3 reaches 709 tokens/s on low-concurrency decode, against 450 tokens/s for our GB200 baseline, both with DSpark speculative decoding. To our knowledge, this is the first TPU inference megakernel. The whole model runs in a single Pallas kernel, and without spec decoding it is roughly 1.4 to 2x the GB200 baseline at batch sizes 1 through 8. We are open sourcing it today. 1/2