How I Used AI
The ideas, framework and product decisions are mine. Claude helped me research, structure, build and tighten them.
I used Claude Code for nearly everything, with Wispr Flow for dictation.
How it’s built
A fully hosted web app at makemultiverse.com. Claude Code wrote and deployed all of it.
The music model on a GPU
DiffRhythm (open source) in my own container on a RunPod serverless GPU, changed to take a seed so the same seed gives the same song. That’s what makes “Keep the tune” work. Early versions ran on Replicate.
Backend and data
A Python function on Vercel makes songs, runs the word check, shares and exports, with ffmpeg for audio. Supabase handles email sign-in, the database (projects, credits spent on the server with a daily cap, word checks, shares) and storage for songs and recordings.
Other models
Whisper large-v3 on Replicate checks every song’s words against the lyric. Claude (Anthropic API) powers the writing help. MuQ-MuLan, inside DiffRhythm, turns a sound clip or description into the model’s style input.
Front end and hosting
Plain HTML, CSS and JavaScript with no framework, styled with the Stageshow design kit. Hosted on Vercel with its own domain, code on GitHub, and evals built into the app.
Promo video
Runway for the animated shots and ElevenLabs for the music and sound, cut together by Claude Code.
How I used AI at each step
Understanding the paper
I read the DiffRhythm paper, then used Explain Paper to break it down in plain terms, and had Claude make short explainer videos to help me understand how the model works.
Choosing the idea
I came in with my own starting ideas and used Claude to brainstorm more and to grill me on which ones were good or bad. My ideas were the seed; Claude helped me test them against what the model can and can't do.
Designing the product
I designed around the model's limitations and fed Claude inspiration from music editing tools (take lanes in Logic Pro and Ableton) to land on the lanes and forking UX. I used the Stageshow skill to build the components and design system.
Building the prototype
Claude Code built the app, deployed DiffRhythm on a RunPod GPU, and exposed the model's seed so "Keep the tune" works.
Evals
I wrote the evals by hand, then used Claude to run them and show results on the Evals page.
Landing page
I gave Claude my product brief plus a few landing pages I liked, and it built the landing page from them.
Promo video
Claude Code generated the video, using ElevenLabs and Runway under the hood. I dictated the direction I wanted with Wispr Flow and went back and forth on the copy.
Write-up and this page
I dictated my thinking with Wispr Flow and gave Claude the structure I wanted. Claude researched and linked sources, and helped me tighten the writing in my own voice.
Agent setup
An orchestrator agent
Helped me figure out how to set up the whole product and coordinated the rest.
Subagents for specific parts
One each for the product brief write-up, the promo launch video, the landing page and the RunPod GPU deployment.
A brainstorming agent
That I used to generate and pressure-test ideas.
Before and after
The framework
"For me, it's something that works today, even with the limitations of the model, but also can get exponentially better as the model gets better… a TAM that will continue to grow with the model getting better… latent demand from just general model feedback of what people are trying to do… from Hacker News and GitHub issues."
A five-part framework (take advantage of the breakthroughs, work today and turn limitations into strengths, show latent demand, have a large market with propensity to pay, get better as the model gets better), with quotes from the DiffRhythm Hacker News thread and GitHub issues as evidence.
The user journey
"We have a concept of a project where I can easily come in… choose from reference audio… or allow the person to come with their own audio… If I like one of the variations of the songs, I can fork it… I can keep the tune… we actually end up using the same seed."
The six-step journey (start a project, pick a sound, hear a round, see which ones are clean, build on a favorite, keep, export and share), each shown with a clip of the working app.
The ideas I ruled out
My notes on language learning, a kids' singing device, fine-tuning for labels, affirmation songs, religious music and gift songs, each tested against the same questions.
The "What other ideas did you consider?" FAQ, with every figure checked and linked. Four numbers didn't hold up and were corrected or dropped.
Where I pushed back
Claude's first drafts weren't always right, and steering it was a big part of the work:
Voice: I installed two writing-review skills (humanizer and unslop) and had Claude rewrite against a sample of my own past specs.
Metrics: We replaced it with a simple rate, like 3 in 10 songs coming out clean.
Accuracy: Claude checked the real app and corrected the copy.
North star: We moved from a percentage to a count of weekly writers who export or share a song.
Skills and rules I set up
Songwriter copy skill
A skill I wrote so every word in the app is plain language for songwriters, with no "generate", "take", "seed" or "GPU" in the interface.
Stageshow
For the design system and components.
Humanizer and unslop
For keeping the write-up in my voice.
Standing rules in Claude's memory
Follow the songwriter copy skill, and commit and push work to GitHub as it lands.
Supporting evidence
- The Evals page: app.makemultiverse.com/evals, with the evals I wrote and Claude runs.
- The seed experiment: the same seed gives back the same song (0.9998 correlation), which is what made "Keep the tune" possible.
- The working product: makemultiverse.com, and the user journey clips in the product brief.