Simon Mo
@simon_mo_
building @inferact for @vllm_project
Joined July 2018
368 Following 4.2K Followers
🫡 vLLM is your engine of choice for agentic workload. The pareto curve is easily understood; but very hard to optimize against. The @inferact team did amazing work here, please check it out (and join us!)
Show more
New blog is out: vLLM x AgentX: Optimizing for Real-World Agentic Serving.
Agent traffic stresses every layer of the serving stack at once. This post walks the full-stack work for optimizing vLLM on Agentic workloads, including the architecture, framework, and runtime optimizations, measured on AgentX, @SemiAnalysis_'s public agentic benchmark.
🧵1/6
Show more
