登録して招待リンクを共有すると、動画再生報酬と紹介報酬を獲得できます。

Elastic
@elastic
参加 October 2009
0 フォロー中    0 ファン
"It feels slow" is the worst ticket a self-hosted LLM generates. vLLM starts, serves, and reports success whether it's configured brilliantly or wastefully. Nothing tells you which. But the answer is already sitting on /metrics: latency split by phase, cache hit rates, batch occupancy, completion outcomes. Almost nobody scrapes it. On a single A10G, 6 metric checks turned "it feels slow" into a verdict: TTFT: 51 ms against a 300 ms p95 target Queue time: 0.01 ms Decode: 97% of total latency Prefix cache: 32% of prefill work skipped entirely The server was over-provisioned, not under. The slowness lived somewhere in front of it: the app, the gateway, or the prompt. Tuning an LLM server is reading telemetry and reasoning about saturation. SREs have been doing that for 20 years. Full walkthrough with the ES|QL queries, the DCGM setup, and all 6 metric checks on an A10G:
もっと見る