AgentX - InferenceXv3: Does CUDA Moat Hold up in Agentic Inferencing? $3 Million USD dataset open sourced, 1 Mil+ Context Length, Multiturn, Sub Agents 95%+ KVCache HitRate, GB300 NVL72, MI355, B200

Currently this is just an InferenceX Preview submission, though we are excited for the full public vLLM Rubin NVL72 InferenceX submission coming soon! Excited for @sgl_project Rubin NVL72 submission coming soon too! Great work to NVIDIA, vLLM, Inferact, the Ex-CentML folks now at NVIDIA, Nick Comly, Xin Li, Alex Sette, Kedar Potdar and team for this amazing work! Link to Dashboard:

ALERT🚨🚨: On apples-to-apples InferenceX Comparisons, TPUv7 Ironwood achieves 50% better perf per dollar than Blackwell Ultra on the new TorchTPU external inference stack! TorchTPU brings native PyTorch to TPUs, & along with Google open-sourcing a bunch of their Pallas inference kernels, Google has laid out a solid foundation for TPU to rapidly externalize. We at SemiAnalysis strongly believe that TPU externalization is heading in the right direction and moving full steam ahead.

TPU Inference Externalization Full Steam Ahead - InferenceX, Up to 50% Better Performance per Dollar, Rapid Externalization of TPU stack, Growing Customer Base, Ironwood, TPUv8i, Reducing CUDA Moat

Proud to see Mooncake featured among the supporters of @SemiAnalysis_ InferenceX initiative. Reliable, transparent benchmarking is critical as inference systems become increasingly disaggregated and complex. We’re excited to contribute to this ecosystem and help push the frontier of efficient, scalable AI inference. More details: #InferenceX# #AIInference# #Mooncake#

Day-0 Inference performance for M3.1 is out! AMD MI355X puts up great results owing to AITER's FlyDSL fused MoE and other fused kernels. shoutout to the AMD and SGL team! Keep track of live updates at InferenceX.

How GLM5.3 Sparse Attention Affects HBM Memory Usage GLM-5.3, KV Cache Offloading, HiSparse, AgentX TileRT, InferenceX DeepSeek Sparse Attention, IndexShare, Single-rollout Asynchronous Optimization, Cybersecurity

DRAMA ALERT🚨: In the YouTube comment section of the NVIDIA Chief Scientist Computer Museum panel, OpenAI Head of Jalapeno replies, saying that Jalapeno has already demonstrated on the SemiAnalysis InferenceX benchmark that Jalapeno is already running very fast (which, btw, beats NVIDIA chips like July Rubin) on non-OpenAI models, refuting the NVIDIA Chief Scientist’s misunderstanding about Jalapeno only being optimized/codesigned to run OpenAI-specific models. InferenceX is the established industry standard, endorsed globally, including by Meta, Oracle, Microsoft, Moonshot, NVIDIA, Weka, vLLM, SGLang, dg.

Engrams Embedding Entendre: Codesign for Efficient DRAM/SSD Offloading New Model Architecture Implications for TAM of DRAM/NVMe, DeepSeek V4.1 Flash, AgentX, InferenceX, NVMe experiments