Register and share your invite link to earn from video plays and referrals.

SemiAnalysis
@SemiAnalysis_
38 Following    177.4K Followers
BREAKING NEWS🚨: RUBIN NVL72 HITS 862 tok/s/user on GLM5.3 744B on agentic workloads! In less than 8 days, NVIDIA has increased interactivity on Rubin by close to 2x through low-latency specialized kernels.
Show more
🚨🚨ALERT: ANTHROPIC IS A/B TESTING USAGE LIMITS Many users on X are reporting LOWER usage limits right now. THEY ARE NOT WRONG. We detected an A/B test going on that DECREASES your monthly API-value by ~20% while INCREASING your 5h usage limits by ~20%. This only affects SOME accounts. (1/2)🧵
Show more
0
162
2.6K
143
Forward to community
Japan’s wave of corporate breaches is the latest sign that AI is changing how hacking works. Since Sept 25, 18 Japanese companies have disclosed breaches, including Tokyo Metro, Japan Post, Lawson, Nikkei, Yamato and Sagawa. Attackers got about 6.6M Times Car accounts, some with driver’s license images. Burger King Japan says 11.18M app member records were affected, and Sourcenext says up to 22.5M records may be out. Last week, JPCERT/CC, the country’s national computer emergency response team, posted its findings from the investigations into these breaches. The attackers used AI to scan for known vulnerabilities and to attack companies’ APIs. Often they did this by pulling apart a company’s Android app to find the endpoints and any keys left in the code. It is also no surprise that Japanese companies are particularly vulnerable. Japan is still catching up on basic digitization, with many offices still using fax machines to this day! Now even the PM is getting involved, urging anyone running a PHP website to patch two critical vulnerabilities immediately, or temporarily take the site offline if they can't.
Show more
皆様に、今夜に対応して頂きたい大至急のお願いです。   多くのウェブサイトで使用されているPHPに関し、PHPのセキュリティチームが、重大な2件の脆弱性に関する注意喚起を行っています。   公開しているウェブサイトでPHPを使用しているか確認し、今夜、10月21日午後11時(日本時間)に公開される修正パッチについて、可及的速やかに適用できるよう準備をお願いします。   修正パッチを速やかに適用することが困難な場合は、例えばウェブサイト等を一時的に停止させることも、リスクを低減させるための有効策の一つと考えられます。   国家サイバー統括室のXアカウントでも関係する情報を発信していますので、そちらも参考にして下さい。 国家サイバー統括室(注意・警戒情報)(@cyber_forecast) / X
Show more
Today we launch SGLang-TorchTPU, built with @Google TPU team and Meta @PyTorch team. We’re excited to see it powering production inference at Meta! SGLang-TorchTPU is a TPU plugin for SGLang that brings: - Top open models on TPU: Muse Spark Open, GLM-5.3 Flash, Kimi K3, Ling 3.1 Flash, MiniMax H3 - SGLang’s production features for large agentic workloads - Custom MegaKernel: Muse Spark Open hits 770 tok/s decode at bs=1 on InferenceX 8K/1K SGLang-TorchTPU is available now:
Show more
WHAT 1,730 CRYSTAL PULLERS TELL US ABOUT MUSK’S 20 GW PLAN: Musk says SpaceX has a “highly proprietary” plan to add 20 GW of power by end-2027, beyond just gas turbines. Solar is a part of it and things are moving. China shipped 1,730 crystal pullers to the US in Jul–Sep 2026, vs. 15 in the prior 18 months. Pullers grow the silicon ingots that are sliced into wafers for solar panels. (1/3)🧵
Show more
BREAKING NEWS🚨🚨: TODAY META CONFIRMS THEY ARE RUNNING TPU INFERENCE IN PRODUCTION. What does this mean for the CUDA moat? For those who read our public newsletter, this is not a surprise, as on Nov 28, 2025, we publicly wrote that Meta is a TPU customer & has lots of engineers working on TPUs. Our accelerator supply chain model has details about Meta's TPU procurement!
Show more
After eight long years, TSMC is finally shrinking their 6T High Density SRAM bitcell for their A14 process! At IEDM, they will present the world's smallest bitcell, <0.017μm2 on their 2028 node. TSMC has been stuck at 0.021μm2 since N5 in 2020. With designs utilizing even more SRAM as memory bottlenecks grow, a 25% denser SRAM with A14 is significant.
Show more
BREAKING NEWS🚨: NVIDIA ENGINEERS AT THE PYTORCH CONFERENCE JUST SAID THAT THE SPEED OF OPTIMIZING CUDA SOFTWARE WITH FRONTIER AI MODELS HAS INCREASED RAPIDLY October 2025 to August 2026: 5x in 1 year with expert human optimization September 2026 to October 2026: 2x in 1 month with AI assistance October 2026: 1.5x with AI orchestration + MegaKernels
Show more
TorchTPU shows Google finally embracing open-source and offer serious support. As our InferenceX shown recently, Google engineers and external companies have demonstrated great TCO.
PyTorch 🤝 Google TPUs. TorchTPU is officially open source! 🚀
TokenSpeed Kernel at today’s Triton Conference poster session! Portable APIs and high-performance kernels for multi-silicon LLM inference. The AMD backend is fully rewritten in Triton + Gluon for ease of use and speed. 👇 ⭐️
Show more
ALERT🚨: SGLANG JUST MERGED MEGAKERNEL SUPPORT, GETTING CLOSE TO 1000 tok/s/user on the GB300 NVL72 AgentX Harness & increasing interactivity by 1.5x on MI355X too! Great work by the SGLang, RadixArk, NVIDIA, and AMD teams! Best of all, this already works out of the box on SGLang Docker images! Excited to rapidly get this on the official AgentX!
Show more
Introducing SGLang Low-Latency Mode, built with @NVIDIAAI and @AMD. On DeepSeek-V4.1-Flash with DSpark, it delivers up to 1.38× P90 per-user token speed on NVIDIA GB300 (671 to 928 tok/s) and 1.56× on AMD MI355X (332 to 516 tok/s) in @SemiAnalysis_'s AgentX trace replay. Highlights: - NVIDIA: overlapping kernels cut decoder layer time from 96.8 to 68.4 µs - AMD: 2 persistent kernels per layer replace dozens of small ones, with cross-GPU communication overlapped inside them. - New plugin system: custom kernels plug in from external libraries, with automatic fallback. - Low-latency kernels for GLM-5.3, K3, and MiniMax M3 are in progress and will plug in next.
Show more
SemiAnalysis is an active supporter of open source in the ML community! As such, we are excited to sponsor the open-source SGLang Summit! SGLang is one of the engines that power production inference across many frontier labs globally! At the summit, we’ll share more about our collaboration with SGLang, including our latest AgentX benchmarks.
Show more
Proud to welcome our newest sponsors to SGLang Summit. In just one week since our public announcement, our sponsor lineup has grown from 11 to 36. Thank you to every sponsor helping bring this community together. Join 2,000+ engineers, researchers, and builders from across the AI ecosystem for technical sessions, hands-on workshops, and the GPU Expo. Hear from Lip-Bu Tan, Lilian Weng, Soumith Chintala, Aravind Srinivas, and more. Meet the teams building across AI hardware, infrastructure, models, and applications. 🗓️ November 12–13, 2026 🌉 Fort Mason, San Francisco 🎟️ Get your tickets:
Show more
ALERT🚨: NVIDIA has been working overtime, nights, and weekends after being shocked 🤯 by last week’s TPUv7 FP8 submission showing better performance per dollar. Unfortunately for NVIDIA, its first GB300 FP4 counter-submission is still 1.64x worse in performance per dollar than TPUv7 for external customers across most of the end-to-end performance curve. The x-axis of the graph below uses end-to-end latency, so a higher curve is better. The TPUv7 curve has higher perf per $ across most of the E2EL Pareto frontier. This initial submission looks like NVIDIA’s first attempt to close the gap, but the gap is still massive. We expect more counter-submissions from NVIDIA as they continue trying to narrow the difference.
Show more
Datacenters 2027: The Year of Acceleration, +50GW in a Single Year 60GW Under Construction, SemiAnalysis Leading Indicators, 1.3T of Lease Contracts Signed YTD, The Shortage Could Move to Chips
Show more
BREAKING NEWS🚨: NVIDIA has just achieved 423 tok/s/user 🚀 on the 2.8T Parameter Kimi K3 model via a new upstream vLLM optimization. 1.68x Faster on the Same Hardware! This was done by NVIDIA CentML's Benjamin, who wrote NVIDIA's new decode-specialized kernels in Cute DSL via aggressive Megakernel fusion. This is a 12,000 LoC Megakernel and is the future of advanced AI-assisted kernel generation. NVIDIA is already working on DeepSeekv4.1 and other models for vLLM too. Someone should give this NVIDIA 10x engineer a raise for this great new major feature!
Show more
TOKENMAXXING: @vllm_project and @inferact achieve 1.3 BILLION tokens per $ of TCO (hyperscaler volume pricing) via disaggregated serving on AgentX. This result is 4x better than the B300 aggregated configuration.
Show more
NVIDIA is not the only company that applies pressure tactics at their events
Amused that AWS just unilaterally cancelled Cloudflare's Reinvent booth. There are always signs when someone finally realizes they're losing. Guess we'll just attend and throw the most fun side events without paying Amazon! Reply if you want an invite.
Show more
MEME: If GPT-7 is AGI, OpenAI could potentially ditch Broadcom entirely and bring everything in-house, as GPT-7 could be better than Broadcom at designing 224G SerDes, handling PD, and managing the supply chain.
Show more
Moratoriums aren't killing the US datacenter buildout. Our analyst @mayabarkin finds only ~2.3 GW of ~21 GW in moratorium areas (excluding the TCEQ pause in Texas) is likely to be delayed, as cited in The Economist. For more, read her report in the SemiAnalysis newsletter👇️
Show more
A few reflections from travelling (w/ @ed_shackle, @sebwride++) through data-centre country in Michigan/Ohio in my latest for @TheEconomist. 1/ The issue really is moving votes to the Democrats. That's now visible in our polling, too (h/t @OwenWntr), but also in the scale of on-the-ground discontent. I'm quite a bit more bearish on the prospects for big new data-centre projects in blue states after the trip. 2/ I really didn't hear that much direct anti-AI sentiment. The mood much more was a cocktail of worries about local harms (water, electricity bills, house prices), plus a total distrust of businesses and politicians. Against that backdrop, assurances about the water usage of closed-loop cooling systems really don't cut through. And, though developers will doubtless get cannier at this, cash payoffs to communities can easily look like subterfuge and do little to help. 3/ Still, moratoriums are at this stage much more bark than bite (h/t @mayabarkin for the data there). Only about 2GW seems actually delayed. The bigger risk is deterring newer projects, but I'd still guess that affects the exact locations more than whether the build-out happens at all. 4/ The question I'm left with is what comes after data centres as the big AI political issue into next year. A few obvious candidates are: a big cyberattack, the sheer scale of AI-related PAC spending in the 2028 cycle, the early innings of job displacement. 5/ What the data-centre issue should tell us, I think, is that voters' hostility to new technologies that change their lives in unpleasant ways is vast—especially when those changes are less positive-sum than data centres. So, however mad you expect AI as a technology to make the next few years, scale that up accordingly to account for a much more scrambled politics. Do take a look at my piece on this here: And at our Checks and Balances podcast, where I contributed a short diary featuring lots of conversations with voters
Show more
The agent access war has started! Meta's Muse recently hit #1# on the App Store. But soon Amazon blocked it. Jefferies then found a third of the US shopping and travel sites it tested block it too. ByteDance is running into the same thing in China. It gave every app 30 days to opt out of its Doubao phone agent. 155 did, including WeChat, Taobao, Alipay, and even DeepSeek and Kimi. What is this war really about? Data security, privacy, or money? An agent shopping for you doesn't click sponsored listings, while Amazon made ~$68B from ads last year. Tencent and Alibaba, which make up 57% of ByteDance's list, are each pushing their own AI agents. Shopify let Muse in and Meta is already signing connector deals with Walmart and Best Buy. If access ends up going to whoever pays, do we eventually need some kind of agent neutrality rule? What do you think?
Show more