vLLM Office Hours #
58# recording is up:
- What's new in vLLM v0.29 and v0.30
- A new hardware backend from Tenstorrent
- An intro to llm-d Semantic Classifier, a lightweight classification service for high-throughput routing
- Why it isn't trying to be a guardrail
Watch + see slides: