Applied AI· riffd · riffdlabs.com

Riffd

Open Riffd ↗
The Problem

Music apps let you listen. Theory apps teach you concepts on a whiteboard. Neither one takes a song you already know and tells you how it actually works. That was the gap.

What I Did

I designed a pipeline that separates any song into its individual parts, identifies the key and chords, and generates a plain-English explanation of the theory behind it. The main coordination challenge was that the three AI models doing the analysis all finish at different times. I designed the system to stream results as each layer completes (instruments first, then chord analysis, then the theory breakdown) rather than making users wait for everything at once.

I also brought in five external services to pull song metadata, lyrics, and context into the output. Playback runs entirely in the browser, so the server is out of the loop once analysis is done.

Live at riffdlabs.com.

Riffd key analysis: circle of fifths, diatonic chords, and common progressions
Key analysis: circle of fifths, diatonic chords, and common progressions
Riffd mixer: stem separation with per-track volume, mute, and solo controls
Mixer: stem separation with per-track volume, mute, and solo
Stack
Demucs (Meta)
Audio source separation on cloud GPU: isolates vocals, bass, drums, guitar, piano, and other; a second melodic pass sharpens harmonic clarity.
Basic Pitch (Spotify)
Converts separated stems into MIDI-equivalent note events, the foundation for per-instrument chord extraction.
Essentia
Key and tempo detection straight from the raw audio signal, independent of stem separation.
Claude API
Synthesizes key, chords, and progression data into a plain-English theory explanation specific to the song.
Web Audio API
Client-side mixing engine handling all stem playback, muting, soloing, and volume; the server is out of the loop after processing.
Flask + Vanilla JS
Lightweight Python backend with a concurrent job queue and progressive result streaming; zero-dependency frontend; deployed on Render.
Open Riffd ↗ ← All work