Reverb

Two AI models. One prompt. Eight seconds of sound. Audible, visible proof of the upgrade.

Reverb sends the identical prompt to two model slots —

"Write JavaScript that plays an 8-second piece of music using ONLY the Web Audio API. No libraries, no external files, no samples. Output only code."

— runs each reply inside a sandboxed iframe with a real AudioContext, paints a live spectrogram per model on Canvas 2D from real AnalyserNode frames, then plays both pieces simultaneously as a duet. The visual is two scrolling spectrograms; the payload is the sound.

Everything is really synthesised and really analysed live. Nothing is mocked, faked, or pre-rendered.

• What you see when you run it
• Tech stack
• Architecture
• The three tricks that make it real
• Providers & capability rules
• Getting started
• Verification: proving it is real
• Project structure
• Contributing
• Feature ideas

1. Type a prompt (or pick a preset: a slow ambient drone, a hard techno loop, a lullaby in a minor key, a sci-fi door opening).
2. Press COMPOSE — both slots are called concurrently with the identical prompt (+ a fresh nonce).
3. Each model's code is armed in its own sandboxed iframe; an offline render immediately produces downloadable WAVs and the spectral fingerprint.
4. Press PLAY A, PLAY B, or PLAY DUET — the spectrograms scroll live (cyan for A, magenta for B), each with a waveform line, RMS meter, and elapsed readout.
5. Press COPY RESULTS AS JSON — machine-checkable proof that both WAVs differ, neither is silence, and both spectrograms received distinct live frames.

The canonical demo: preset "a hard techno loop" — both panels must show a periodic vertical pattern (a beat). A flat panel means that model's code never ran.

┌─────────────────────────── Browser · localhost:5173 ───────────────────────────┐
│                                                                                │
│  React UI (App.tsx)                                                            │
│  ├── prompt + 4 presets ──────────────► POST /api/compose (Vite /api proxy)    │
│  ├── per-slot provider config ────────► localStorage (keys never hit the repo) │
│  │                                                                              │
│  ├── iframe A (sandbox="allow-scripts", srcdoc)   iframe B (same)              │
│  │     ▲  postMessage {type:'run'|'render'|'stop', code}                       │
│  │     └─ postMessage {type:'frame'|'ended'|'rendered'|'error'} ─► Canvas 2D   │
│  │                                                                              │
│  ├── live path:  AnalyserNode frames ──► scrolling spectrogram (offscreen +    │
│  │                getByteFrequencyData/TimeDomainData     drawImage @ 60fps)   │
│  └── render path: OfflineAudioContext PCM ──► WAV encode · duet mix · FFT      │
│                   fingerprint (RMS, centroid, peak)                            │
└────────────────────────────────────────────────────────────────────────────────┘
          │ /api/*                                  ▲ OpenAI-compatible HTTP
          ▼                                         │
┌─────────────────────────── Node backend · localhost:3001 ──────────────────────┐
│  POST /api/compose                                                             │
│   • identical prompt → both slots concurrently (Promise.all)                   │
│   • fresh random nonce per call; prompt reuse = hard 409                       │
│   • max_tokens ≥ 900 floor, no cap (up to 128000); empty content → escalate   │
│     ×2 → 16k → 64k → 128k before giving up, with a diagnostic error           │
│   • chat_template_kwargs ONLY for Particle.ai deepseek-* models                │
│   • reasoning_content stripped before anything is logged or returned           │
│   • code extraction (fence → fallback) + sha256 + NO_AUDIO rejection           │
│  POST /api/models  → live model-list proxy (keeps local providers CORS-free)   │
└────────────────────────────────────────────────────────────────────────────────┘
        │                    │                    │                 │
        ▼                    ▼                    ▼                 ▼
   Particle.ai           Ollama               LM Studio      OpenRouter / Custom
   (key required)    127.0.0.1:11434/v1   127.0.0.1:1234/v1   any base URL

Each iframe is a full HTML document built from a srcdoc string (src/sandbox.ts). It has no same-origin, no parent DOM access, and no network beyond what the page itself grants — it can only talk back over postMessage:

parent ──► { type:'run',    code }   live playback with a tapped AudioContext
parent ──► { type:'render', code }   OfflineAudioContext render → PCM back
parent ──► { type:'stop' }           cancel rAF loop, close context
iframe ──► { type:'ready' }
iframe ──► { type:'frame', freq:Uint8Array(1024), time:Uint8Array, rms, t }  (~60/s)
iframe ──► { type:'ended',  durationMs }
iframe ──► { type:'rendered', sampleRate, duration, length, channels:Float32Array[] }
iframe ──► { type:'error',  phase:'eval'|'audio'|'render', error, stack }

Model code connects its nodes to ctx.destination — and we must both hear it and measure it. Reverb replaces the constructor with a wrapper that inserts a gain sink + analyser between the code and the speakers:

function Tapped() {
  var ctx = new Base();                        // real AudioContext
  var an = ctx.createAnalyser();
  an.fftSize = 2048;
  an.smoothingTimeConstant = 0;
  var sink = ctx.createGain();
  sink.connect(an);
  an.connect(ctx.destination);
  // the model code connects to ctx.destination; hand it the sink instead
  Object.defineProperty(ctx, 'destination', { get: function () { return sink; } });
  ctx.__an = an;
  return ctx;
}
window.AudioContext = Tapped;

The parent then paints each frame message as a new column at the right edge of an offscreen canvas and blits with drawImage — the scroll stays at 60fps regardless of panel size.

For WAV export and the spectral fingerprint, the same code is re-evaluated with AudioContext replaced by a Proxy around a real OfflineAudioContext(2, 44100×8, 44100):

function Shim() {
  var ctx = new RealOAC(2, LEN, SR);
  return new Proxy(ctx, {
    get: function (target, prop) {
      if (prop === 'state') return 'running';
      if (prop === 'resume' || prop === 'suspend' || prop === 'close')
        return function () { return Promise.resolve(); };  // code may close() — ignore it
      var v = target[prop];
      return typeof v === 'function' ? v.bind(target) : v;
    },
    set: function (target, prop, value) { target[prop] = value; return true; }
  });
}

currentTime stays 0 during evaluation, so code that schedules at ctx.currentTime + 0.1 lands exactly where it should; startRendering() then yields genuine PCM, which the parent encodes to 16-bit WAV and analyses with its own radix-2 FFT (RMS, spectral centroid, peak frequency).

Provider response caches silently fake determinism. Every prompt leaves the backend as:

${userPrompt}\n\n[nonce:${randomBytes(8).toString('hex')}]

and every prompt SHA-256 is recorded — a repeat is a hard error. Two runs can never share a prompt, so "the new model is better" can never be a cached reply.

• Capability is detected, never assumed: reasoning tokens are shown only when the provider actually reports them; otherwise n/a and the thinking toggle hides. Never 0, never invented.
• The model picker is populated from /models and always accepts hand-typed names (deepseek-v4-flash-0731 responds even when it is absent from Particle's list). A failing /models never gates a run.
• Errors surface the provider's real text, e.g. Cannot reach http://127.0.0.1:11434 — is Ollama running? (ECONNREFUSED).

git clone <your-fork> reverb && cd reverb
npm install
npm run dev          # backend :3001 + web :5173, one command

Open http://localhost:5173, paste API keys in the slot config (persisted to localStorage, forwarded per-call to the backend — nothing is hardcoded in the repo), pick providers, COMPOSE, then play.

Other scripts:

npm run typecheck    # strict tsc for web + server
npm run verify:api   # 27 end-to-end backend checks against a fake provider
npm run build        # typecheck + vite build

Works with zero keys if you run Ollama or LM Studio locally — select the preset, and the model list fills itself.

npm run verify:api spins up a fake OpenAI-compatible provider and asserts, among others:

• fence extraction works and the code contains AudioContext
• reasoningTokens is 7 when usage reports it, null (→ n/a) when it does not
• the CoT text (reasoning_content) never appears anywhere in the response
• empty content triggers exactly one doubled-budget retry
• identical prompt text across two runs still yields different prompt SHA-256s (fresh nonce)
• a dead Ollama produces the exact "is Ollama running?" message

In-app, COPY RESULTS AS JSON emits the same class of proof for real runs: wavsDiffer, wavsNotSilence, spectrogramChangedA/B (rolling frame checksums), frames counted, sandbox errors with stack lines.

reverb/
├── server/src/
│   ├── index.ts          Express app: /api/compose, /api/models, /api/health
│   └── compose.ts        slot calls, nonce guard, extraction, stripping
├── shared/
│   └── providers.ts      provider presets + prompts (used by both sides)
├── src/
│   ├── App.tsx           UI: states, transport, config, JSON export
│   ├── sandbox.ts        srcdoc iframe + postMessage protocol
│   ├── Spectrogram.tsx   Canvas 2D scrolling spectrogram (imperative, 60fps)
│   ├── wav.ts            WAV encode, duet mix, radix-2 FFT fingerprint
│   ├── api.ts            typed client for the backend
│   └── types.ts          shared UI types
├── scripts/
│   └── verify-api.mjs    end-to-end backend verification

Forks and PRs are welcome.

1. Fork → branch (feat/my-feature) → npm install → npm run dev.
2. Keep the honesty invariants: no mocked audio, no canned frames, no invented numbers. If your feature touches analysis or playback, extend scripts/verify-api.mjs.
3. npm run typecheck && npm run verify:api must pass.
4. Open a PR describing what a viewer of the screen recording would notice.

Good first issues: better error surfacing for exotic provider payloads, a11y passes on the transport controls, more colour maps.

• Tournament mode — bracket N models pairwise; winners advance by human vote or by fingerprint score.
• Remix chain — feed Model B the code from A plus "improve this" and hear the delta.
• Spectrogram diff view — A/B overlay mode with per-bin difference highlighting.
• Shareable runs — encode slot config + code into the URL; render server-side for OG audio embeds.
• More fingerprints — MFCCs, stereo width, onset density; render them on the card.
• Docker Compose — one command for app + Ollama with a pulled model.
• Vitest unit tests — WAV encoder, FFT, extraction edge cases.
• Live code editing — tweak the model's output in an editor pane and re-run instantly.