Still early, but I wanted to share a quick demo of the new hands-free music-making application I built while testing GPT-5.6 Sol. Iโm also applying to work with
@gdb and the team as an early GPT-Live API design partner.
Like any new instrument, Iโll need some practice! Speaking fluidly back and forth with Codex in my DAW feels like working in a studio with a recording engineer. Here is what it can already do. Iโll share more demonstrations over the next week.
Built for macOS in Swift and C++. It can edit MIDI, audio, video, notation, tempo, meter, and harmony layers. It uses gpt-realtime-2.1, Whisper, and Codex App Server to collaborate with an assistant that can already perform more than 300 actions.
There is also an iOS companion application that allows me to work remotely on my recording studio computer from anywhere, with voice mode still active.
As a lifelong composer, I already keep the entire editing surface and instruments in my mind. I can imagine a piece of music and how to make it in software faster than I can build it manually. This removes that friction and will become very fast for me with practice.
It also has long-term project memory, deep music-analysis tools, a found-sound synthesizer builder, and access to my catalog of more than 550 tracks for sampling and licensing. A companion ChatGPT/Codex plugin is coming soon.
I hope to launch in the next few weeks! Hugely inspired by
@mcleavey,
@guinnesschen,
@athyuttamre, and the entire audio and voice team at
@OpenAI ๐