The numbers below are the gate this exact deploy went through — not a badge from some earlier good day.
A service with no backend still has plenty to break: audio parsing, a cross-origin engine import, an LLM API, a CDN's caching rules. One test layer cannot cover that spread — three small ones can.
Pure-logic tests of the app modules: audio content-sniffing, config, the chat tool registry's validation, the declared workflow machine, the debug store. Fast enough to run on every change.
The same modules re-tested under REAL browser semantics — genuine File/Blob objects, genuine localStorage, genuine custom-element upgrade for the site nav. The page ships with the site, so you can watch the suite pass against this very deploy, in your own browser: run the browser unit tests now.
Headless Chromium boots the REAL app: the engine imports cross-origin, the full action contract publishes, a real voice-note fixture ingests, the debug and chat panes work. A second suite scripts OpenRouter and drives the whole chat tool loop — including the model genuinely redrawing the infographic.
After the deploy, plain fetches against the live site: the version it claims, every cache-busted asset, the samples and prompts, the engine origin + CORS, and every GitHub/self link on the record pages. The pipeline checks its own work.
The key user journeys — key setup, the one pass, the chat — replayed with a scripted, deterministic model and screenshotted at each story moment. An image-diff gate compares every shot to its committed baseline: pixel noise is ignored, a real UI change either fails the run (a caught visual regression) or refreshes the baseline with the feature — and those screenshots ARE the user docs' images. One run, three outputs.
Some .ogg voice notes reached the model undecodable — and came back as fluent,
confident transcripts of audio nobody recorded. A unit-tested content-sniffer (magic
bytes, never the file extension) now guarantees the model receives real audio, and a
six-variant regression matrix keeps it that way.
Fresh HTML paired with ten-minute-old cached modules broke a new feature silently. Now CI stamps every asset URL per release and the live-QA layer fetches each one — this class of failure cannot ship quietly again.
A disabled OpenRouter key surfaced as a network error (the rejection carries no CORS headers, so the browser hides it). Found live via the app's own debug pane; fixed with a key-status re-check; pinned by an integration test.
Transcribing the same file twice failed with "not an audio file" — the engine silently dedupes identical uploads. The end-to-end test caught it; the fix reuses the deduped queue item; the case is now part of the gate.