Red Hat AI
@RedHat_AI
Accelerating AI innovation with open platforms and community.
The future of AI is open.
Joined May 2018
2.1K Following 12.6K Followers
What does it take to serve a model that talks back, or one that generates video with audio?
Most text models advance one token at a time. These don't, and they need different scheduling to match.
New recap on how vLLM-Omni serves them:
Show more
