Real-time Audio Visualizer · Personal project · 2026
A browser-native music visualizer that maps the timbre of live audio onto a glowing 3D trail using streaming PCA, no build step, no backend, no framework, directed by me and built with heavy AI assistance from Claude, down to a kiosk-ready deployment on a Linux mini-PC.
Most music visualizers flash to the beat or bounce EQ bars, reacting to volume, not to what a sound actually is. I wanted something that listened for timbre instead, the tone colour underneath the notes, and let that alone decide where it went in space. I found the technique in the work of sound artist Lucio Arese: analyse a sound's spectral fingerprint dozens of times a second, continuously fold that down to three dimensions, and let the moving result draw itself. Nothing choreographed, no song structure fed in, just the grain of the sound finding its own shape.
"There's a moment, right before a sound finishes becoming itself, where it's still just potential, a shape it hasn't taken yet. What we've built doesn't visualize music, not in the beat-flashing, EQ-bar sense of the word, it listens for the grain of a sound and lets that grain choose its own place in space. Two sounds that feel alike find each other in the dark without being told to. It's less a visualization than a portrait of listening itself."
I directed and drove the whole build, the architecture, the algorithms, every debugging session, with substantial AI assistance from Claude on implementation. I think that combination, knowing exactly what you want built and why, then working effectively with AI tools to get a genuinely novel real-time system actually shipped, is as much the point of this project as the visuals are.
The full signal chain, audio capture, FFT, mel-frequency cepstral analysis, an online PCA solver, and WebGL rendering, runs entirely client-side in vanilla JavaScript, with a single vendored dependency (Three.js). No build tooling, no framework. It's served as static files and runs identically in a browser tab or a Chromium kiosk shell on a Linux mini-PC, audio sourced from the system's PipeWire monitor rather than a microphone.
Underneath, a hand-specified FFT and 48-band mel filterbank derive 40 MFCCs roughly every 43 milliseconds. A streaming PCA solver, using exponentially-weighted covariance and a periodic Jacobi eigendecomposition, continuously re-tunes its projection basis to whatever's currently playing, adapting to roughly the last 15 seconds of audio rather than needing a whole track up front. The result renders as a glowing, Catmull-Rom-smoothed trail in WebGL, with hue-cycling, responsive, and solid colour modes.
AudioWorklet.addModule() turned out to maintain a module cache entirely separate from the page's normal ES-module graph, one that survives hard reloads. Worklet changes appeared to silently "not apply" until this was identified and fixed with explicit cache-busting.getUserMedia and a complementary-hue cancellation bug in additive blending along the way.gl.lineWidth() is clamped to 1px on almost every real backend. Solved with a screen-space multi-copy technique, the same line redrawn with per-pixel clip-space offsets in a filled-disc pattern.MediaStream (headless Chromium's fake media devices don't resolve getUserMedia) to exercise and screenshot the full pipeline before shipping.
