Cyrus(@cyrusasg):multimodal inference doesn't fit the mold of standard text autoregressive generation. text-to-speech models feature different architectures, different states, and different batch shapes. we rebuilt our tts serving around that and simultaneously reduced our time to first audio while improving throughput by several fold.

Cyrus

@cyrusasg

research lead @decagonai prev @harvard post-training, inference, mlsys

가입 March 2022

255 팔로잉 중 552 팬

Cyrus@cyrusasg

2026.08.20 18:11

multimodal inference doesn't fit the mold of standard text autoregressive generation. text-to-speech models feature different architectures, different states, and different batch shapes. we rebuilt our tts serving around that and simultaneously reduced our time to first audio while improving throughput by several fold.