← Projects

Riffd

The Problem

Music apps let you listen. Theory apps teach you concepts on a whiteboard. Neither one takes a song you already know and tells you how it actually works. That was the gap.

What I Did

I designed a pipeline that separates any song into its individual parts, identifies the key and chords, and generates a plain-English explanation of the theory behind it. The main coordination challenge was that the three AI models doing the analysis all finish at different times. I designed the system to stream results as each layer completes — instruments first, then chord analysis, then the theory breakdown — rather than making users wait for everything at once.

I also brought in five external services to pull song metadata, lyrics, and context into the output. Playback runs entirely in the browser so the server is out of the loop once analysis is done.

Scoped, designed, and deployed entirely solo. Live at riffdlabs.com.

Riffd key analysis — circle of fifths, diatonic chords, and common progressions
Riffd mixer — stem separation with per-track volume, mute, and solo controls

Stack

Demucs (Meta) Audio source separation on cloud GPU — isolates vocals, bass, drums, guitar, piano, and other; second melodic pass for harmonic clarity
Basic Pitch (Spotify) Converts separated stems into MIDI-equivalent note events — the foundation for per-instrument chord extraction
Essentia Key and tempo detection from the raw audio signal, independent of stem separation
Claude API Synthesizes key, chords, and progression data into a plain-English theory explanation specific to the song
Web Audio API Client-side mixing engine — all stem playback, muting, soloing, and volume control; server is out of the loop after processing
Flask + Vanilla JS Lightweight Python backend with concurrent job queue and progressive result streaming; zero-dependency frontend; deployed on Render