Register and share your invite link to earn from video plays and referrals.

Akshay 🚀
@akshay_pachaar
Simplifying LLMs, AI Agents, RAG, and Machine Learning for you! • Co-founder @dailydoseofds_• BITS Pilani • 3 Patents • ex-AI Engineer @ LightningAI
Joined July 2012
512 Following    292.6K Followers
Big moment for Voice AI. A typical voice-agent pipeline has three model stages. Speech-to-text converts the caller’s audio into text. The LLM processes that text and generates a response. Text-to-speech converts the response back into audio. For speech accuracy, information can be lost at two boundaries. - STT must preserve what the caller said on the way in. - TTS must preserve what the LLM generated on the way out. If a caller says “RX-B17,” the transcription model could hear “RX-D17” before the LLM receives it. Even if the LLM generates “RX-B17” correctly, the speech model could still make the final audio sound like “RX-D17.” The LLM’s reasoning accuracy is a separate problem. Both speech layers must preserve the information entering and leaving it. This becomes important with medication names, dosages, appointment times, tracking numbers, and alphanumeric codes. Deepgram built Flux TTS for the speech layer. In an evaluation containing drug names, tracking numbers, dollar amounts, technical strings, and alphanumerics, Flux recorded a 3.4% word error rate. I tested the complete voice loop through a pet-hospital conversation about my dog Biscuit’s prescription refill. The agent had to understand and repeat the medication, dosage, prescription number, and a collection time that changed during the call. The test worked well. The agent retained the changing details, the voice stayed natural throughout, and the exchange felt like a call I would be comfortable having with an actual pet hospital. Build with Flux TTS: For every $1 spent on Flux TTS, Deepgram will match $1 in credits, up to $500. My recording is below, and I worked with the Deepgram team to put this post together.
Show more