Cyrus(@cyrusasg):multimodal inference doesn't fit the mold of standard text autoregressive generation. text-to-speech models feature different architectures, different states, and different batch shapes. we rebuilt our tts serving around that and simultaneously reduced our time to first audio while improving throughput by several fold.

Cyrus

@cyrusasg

research lead @decagonai prev @harvard post-training, inference, mlsys

参加 March 2022

255 フォロー中 552 ファン

Cyrus@cyrusasg

2026.08.20 18:11

multimodal inference doesn't fit the mold of standard text autoregressive generation. text-to-speech models feature different architectures, different states, and different batch shapes. we rebuilt our tts serving around that and simultaneously reduced our time to first audio while improving throughput by several fold.