登録して招待リンクを共有すると、動画再生報酬と紹介報酬を獲得できます。

Akshay 🚀
@akshay_pachaar
Simplifying LLMs, AI Agents, RAG, and Machine Learning for you! • Co-founder @dailydoseofds_• BITS Pilani • 3 Patents • ex-AI Engineer @ LightningAI
参加 July 2012
512 フォロー中    292.6K ファン
As an AI engineer, please learn: - Learn the roofline model and why decode is memory-bound - Deploy vLLM and SGLang, then read their schedulers - Understand paged attention from the code, not the blog post - Build observability before you optimize anything - Track TTFT, inter-token latency, throughput, queue depth - Use Grafana + Prometheus for inference dashboards - Turn on prefix caching and find which workloads it helps - Learn continuous batching and chunked prefill - Run load tests with 1000+ concurrent requests - Report p50, p95, p99, never just the mean - Master quantization tradeoffs (FP8, INT4, AWQ, GPTQ) - Learn speculative decoding and where it stops helping - Set up KV cache eviction for long contexts - Try disaggregated prefill and decode serving - Learn Kubernetes for AI workloads and autoscale on queue depth - Learn how inference costs break unit economics - Build your own model router by cost, latency, quality - Create a token budgeting system per request - Build one inference service and benchmark it publicly - Read inference research instead of model release news - Start sharing your optimization benchmarks I put together a 10-week plan that covers every one of these at 30 minutes a day. It is 50 sessions, split between reading the theory and building on your own service, and all of them feed one artifact: an inference service you deploy, instrument, load test past 1000 concurrent requests, tune, and publish as a reproducible benchmark. It is open on GitHub and contributions are welcome, especially newer sources worth adding. I am working through it myself and will share more content on this going forward, so stay tuned. GitHub repo: (don't forget to star 🌟)
もっと見る