Register and share your invite link to earn from video plays and referrals.

Rohan Paul
@rohanpaul_ai
Compiling in real-time, the race towards AGI. The Largest Show on X for AI. 🗞️ Get my daily AI analysis newsletter to your email 👉
Joined June 2014
6.8K Following    154.3K Followers
New Google paper shows LLM infrastructure optimization does not have to be a giant brute-force search if an agent can understand why the system is slow. The key shift is from “search everything” to “understand the bottleneck, then search the tiny part that matters.” Instead of asking, “Which of these 100 configurations is fastest?”, this work asks, “What is actually causing the slowdown?” That one change dramatically shrinks the search. Its Analyzer Agent reads profiler traces and classifies the bottleneck as compute, memory, or communication, then a Proposal Agent retrieves engineering knowledge and generates three targeted ways to map data, model, and sequence parallelism across TPU chips. Across eight production workloads spanning dense and MoE models, pre-training, post-training, serving, and TPU systems from 2 to 2,048 chips, the human-validated production configuration appeared in PROMPTS' first batch every time. In seven of eight cases, it was also the first configuration tested, and the agent's top-ranked proposal matched what engineers ultimately adopted in 87.5% of cases.
Show more