註冊並分享邀請連結,可獲得影片播放與邀請獎勵。

vLLM
@vllm_project
A high-throughput and memory-efficient inference and serving engine for LLMs. Join to discuss together with the community!
加入 March 2024
37 正在關注    54.1K 粉絲
Thanks to @SemiAnalysis_ for highlighting the vLLM ecosystem’s work on TPU, in particular the Kimi K3 megakernel and vLLM TorchTPU powering AgentX results across GLM-5.2, Qwen3.5 397B, and Kimi K3. Shoutout to @inferact and @googlecloud for pushing vLLM performance on TPU. The @SemiAnalysis_ write-up below is a great read 👇
顯示更多
The Iron(wood) Throne: Google's TPU Stakes Its Claim on Inference Viable Alternative of CUDA Moat, NVIDIA Working Nights & Weekends, Jensen’s Counter Submission, Kimi K3, Meta Muse Spark Open, GLM5.3, TorchTPU, SGLang, vLLM, llm-d, Ray
顯示更多