Register and share your invite link to earn from video plays and referrals.

Guitar Cat + LLM 「Will @ SGLang Summit in SF」
@GenAI_is_real
0 Following    0 Followers
We just tried @MiniMax_AI's M3.1 Flash in our SGLang Omni cascaded voice interaction system, and the experience was genuinely impressive! 🐰🚀 Our original pipeline was Qwen3-ASR → Qwen3 LLM → Qwen3-TTS. We replaced the locally hosted LLM with the MiniMax M3.1 Flash API while keeping the rest of the pipeline unchanged, and recorded a demo to see how it performs in real conversations. The results were excellent. M3.1 Flash responded incredibly fast, understood our questions well, and consistently delivered sharp, concise, and direct answers. The conversation felt remarkably smooth and natural. One thing I increasingly appreciate about voice agents is that intelligence alone isn't enough. Unlike text, users can't easily skim an audio response. A model that understands what you need and gets straight to the point can make a huge difference to the overall interaction experience. This is also what makes the modular architecture of SGLang Omni so exciting to me. We can swap individual components, connect local speech models with frontier LLM APIs, and immediately explore how different models change the end-to-end interaction experience. Huge congrats to the MiniMax team! Really impressive work, and we're excited to explore more together. 🔥
Show more
More good news for Fish Audio Drama 3! 🎉 Thanks to @_akhaliq for the INT4 quantization and his incredible support, SGLang Omni now officially supports Drama 3 INT4! The quantized checkpoint reduces engine weight memory from 37.0 GB to 20.8 GB, with comparable voice-cloning quality in our Seed-TTS EN evaluation. Making powerful speech models more accessible and affordable to serve is exactly what we love working on. We've also been working closely with the @huggingface team to expand our Drama 3 Space with a completely redesigned UI. Even more excitingly, the Space will soon run Drama 3 on SGLang Omni, powered by an H200 generously sponsored by Hugging Face! We're truly grateful for the recognition, trust, and support. Huge thanks again to AK, the Hugging Face team, and @FishAudio for making the model available with open weights. This is what a healthy open AI ecosystem looks like: model builders, inference engineers, and platform teams working together to make great models accessible to everyone. More optimizations coming soon! 🚀 PR: Space:
Show more
Let's see how Open-Source LLM made a 42/42 score on IMO this year!