註冊並分享邀請連結,可獲得影片播放與邀請獎勵。

alexa griffith
@alexa_griffith_
book enthusiast, senior principal engineer at Red Hat, Inference FDE, @alexasinput podcast host
加入 November 2019
537 正在關注    540 粉絲
llm-d flow control, chapter 8: same tenant, different priority. Tenant A at priority 100 and Tenant A at priority 0 have separate queues. Requests with the same fairness ID AND priority share a queue. That pair is called the FlowKey. Try it yourself:
顯示更多