alexa griffith(@alexa_griffith_):llm-d flow control, chapter 2: shared inference under burst pressure. A GPU pool can have spare capacity on average and still run out during a traffic burst. With @_llm_d_ flow control enabled, excess requests are queued until capacity becomes available. https://t.co/8ynwKnOdIt

alexa griffith

@alexa_griffith_

book enthusiast, senior principal engineer at Red Hat, Inference FDE, @alexasinput podcast host

参加 November 2019

521 フォロー中 504 ファン

alexa griffith@alexa_griffith_

2026.09.18 16:32

llm-d flow control, chapter 2: shared inference under burst pressure. A GPU pool can have spare capacity on average and still run out during a traffic burst. With @_llm_d_ flow control enabled, excess requests are queued until capacity becomes available.