Red Hat AI(@RedHat_AI):Great article by @alexa_griffith_. When multiple teams share one GPU pool, arrival order shouldn't decide who gets served first. @_llm_d_ flow control adds admission before routing. Under saturation, priority decides which traffic advances, fairness keeps one tenant from starving the rest, and lower priority work waits. Admission is when a request runs, routing is where, and keeping them separate is the whole idea. Worth a read!

Red Hat AI

@RedHat_AI

Accelerating AI innovation with open platforms and community. The future of AI is open.

가입 May 2018

2.1K 팔로잉 중 12.6K 팬

Red Hat AI@RedHat_AI

2026.09.16 17:58

Great article by @alexa_griffith_. When multiple teams share one GPU pool, arrival order shouldn't decide who gets served first. @_llm_d_ flow control adds admission before routing. Under saturation, priority decides which traffic advances, fairness keeps one tenant from starving the rest, and lower priority work waits. Admission is when a request runs, routing is where, and keeping them separate is the whole idea. Worth a read!