kwindla(@kwindla):If you're interested in voice UI, or music, or both, it's worth watching all of this video that Jon just dropped. Voice-controlled, generative music. The interface will be familiar to anyone who has used a digital audio workstation. But also ... deep integration of voice makes this feel like a very new kind of software. I think it might be that when we've fully figured out the transition to voice-first UIs, we'll feel like the transition from GUI to AI-native, voice-first software was as big as the transition from command-line to GUI software. Jon has shown me videos of various stages of development on this project. We've been talking a little bit about voice UI patterns for consumer software and voice UI patterns for professional software. Both categories are clearly going to become voice-centric. Talking is just so efficient and flexible.

2026.09.02 05:16

If you're interested in voice UI, or music, or both, it's worth watching all of this video that Jon just dropped. Voice-controlled, generative music. The interface will be familiar to anyone who has used a digital audio workstation. But also ... deep integration of voice makes this feel like a very new kind of software. I think it might be that when we've fully figured out the transition to voice-first UIs, we'll feel like the transition from GUI to AI-native, voice-first software was as big as the transition from command-line to GUI software. Jon has shown me videos of various stages of development on this project. We've been talking a little bit about voice UI patterns for consumer software and voice UI patterns for professional software. Both categories are clearly going to become voice-centric. Talking is just so efficient and flexible.

Jon Taylor@JonPTaylor

2026.08.31 21:28

Over a year ago, I posted a video where Gemini and I collaborated on an Ableton Live Session. Music AI is fun! Creating voice-native music software is something I'm really passionate about, so, here is Jamcat. Jamcat's concept is a voice controlled, generative music performance tool, designed to inspire live, in the moment jam sessions. It uses LLMs, with both text-to-audio and audio-to-audio models to create music in realtime, entirely hands-free. ➡️ Pipecat PhoneLLM Alpha 1 for session control, running on @modal. ➡️ "StemGen" routing and workload subagent. ➡️ Foundation 1, for melodic stems. ➡️ Various audio models on @fal (like the excellent @ElevenLabs Music) for vocals, drums, textures and one-shots. ➡️ ... and of course @pipecat_ai! Voice commands can be toggled via push-to-talk (or a foot pedal, if you're wielding an axe), or it can listen the entire time (diarization.) It makes some mistakes, or sometimes misses a beat, but most of the mistakes in this video were me not paying attention to the bar counts, like a true fake musician.