Cyrus(@cyrusasg):multimodal inference doesn't fit the mold of standard text autoregressive generation. text-to-speech models feature different architectures, different states, and different batch shapes. we rebuilt our tts serving around that and simultaneously reduced our time to first audio while improving throughput by several fold.

Cyrus

@cyrusasg

research lead @decagonai prev @harvard post-training, inference, mlsys

Joined March 2022

255 Following 552 Followers

Cyrus@cyrusasg

2026.08.20 18:11

multimodal inference doesn't fit the mold of standard text autoregressive generation. text-to-speech models feature different architectures, different states, and different batch shapes. we rebuilt our tts serving around that and simultaneously reduced our time to first audio while improving throughput by several fold.