Red Hat AI(@RedHat_AI):Great article by @alexa_griffith_. When multiple teams share one GPU pool, arrival order shouldn't decide who gets served first. @_llm_d_ flow control adds admission before routing. Under saturation, priority decides which traffic advances, fairness keeps one tenant from starving the rest, and lower priority work waits. Admission is when a request runs, routing is where, and keeping them separate is the whole idea. Worth a read!

Red Hat AI

@RedHat_AI

Accelerating AI innovation with open platforms and community. The future of AI is open.

加入 May 2018

2.1K 正在关注 12.6K 粉丝

Red Hat AI@RedHat_AI

2026.09.16 17:58

Great article by @alexa_griffith_. When multiple teams share one GPU pool, arrival order shouldn't decide who gets served first. @_llm_d_ flow control adds admission before routing. Under saturation, priority decides which traffic advances, fairness keeps one tenant from starving the rest, and lower priority work waits. Admission is when a request runs, routing is where, and keeping them separate is the whole idea. Worth a read!

显示更多