注册并分享邀请链接,可获得视频播放与邀请奖励。

alexa griffith
@alexa_griffith_
book enthusiast, senior principal engineer at Red Hat, Inference FDE, @alexasinput podcast host
加入 November 2019
537 正在关注    540 粉丝
llm-d flow control, chapter 8: same tenant, different priority. Tenant A at priority 100 and Tenant A at priority 0 have separate queues. Requests with the same fairness ID AND priority share a queue. That pair is called the FlowKey. Try it yourself:
显示更多