註冊並分享邀請連結,可獲得影片播放與邀請獎勵。

Red Hat AI
@RedHat_AI
Accelerating AI innovation with open platforms and community. The future of AI is open.
加入 May 2018
2.1K 正在關注    11.7K 粉絲
New free learning path on the Red Hat Developer Sandbox: compress, serve, and benchmark a model with @vllm_project, hands-on in Jupyter, no GPU needed. Here's what you'll actually do: ⚡ Quantize Qwen3 to W4A16 with LLM Compressor using GPTQ. Measure the result: 42% smaller, 8.2% perplexity increase. Learn to decide if that tradeoff fits your use case. 🚀 Connect to a running vLLM server and send requests via the OpenAI-compatible API. Watch 5 concurrent requests handled in real time. See prefix cache queries increment live via the Prometheus metrics endpoint. 📊 Run a GuideLLM benchmark: TTFT, inter-token latency, and E2E latency at p50, p95, and p99. Run Hellaswag with lm_eval. Cross-reference with the published model card to make a deployment decision backed by numbers. Less than an hour to complete. Free account. Built by @cedricclyburn and Michael Santos. 🙏
顯示更多