Your play button shouldn't know what it's playing

A while ago I wrote about treating an algorithm visualizer as a debugger: a recorded execution you scrub, with the code, the log, and the visual stage pinned to the same instant. That post described one kind of animation — a sorting trace, frozen into frames you can seek through. It is not the only kind I ship, and the other kinds are where this story starts.

Press play on bubble sort and you're scrubbing a recording. Press play on the stack and you're watching a short seeded demo build a structure you then poke at yourself. Press play on the MNIST classifier and a pen draws a digit across a grid while a real neural net, running in your browser, changes its mind in real time about what you drew. Three lessons, three genuinely different things happening behind the same triangle-shaped button.

For a while, that button was three different buttons wearing the same icon. And one of the three didn't exist at all.

Three kinds of play

It helps to be precise about how different these really are, because the differences are what make unifying them interesting rather than trivial.

A timeline lesson — bubble sort, binary search, the depth-first traversal — is a recording. A pure step generator walks the algorithm over your input before anything renders and produces a deterministic array of snapshots. The player is a Remotion <Player>: it owns a frame clock, durationInFrames is known up front, and every visual is a function of the current frame. Scrubbing to frame 212 always draws the same picture because the picture is computed from 212. This is the debugger model, and it is wonderfully well-behaved.

An operation lesson — the stack, the queue, the linked list, a hash table that actually collides — is interactive, but it cheats toward the timeline model in a clever way. It opens on a seeded demo: a short scripted sequence of operations, played exactly like a timeline (paused at frame 0, scrub it, step through it) so you watch the structure get built and watch its signature property show up. Then you start operating on it yourself, and each operation generates its own little trace. Still Remotion underneath. Still a frame clock.

A live lesson has no recording and no frame clock at all. The MNIST classifier is a plain React view. You draw, every stroke runs through the real trained weights, the hidden neurons and output probabilities re-render reactively, and nothing was computed ahead of time because the input is whatever your mouse just did. The training-loop playground is the same shape: a requestAnimationFrame loop runs gradient descent a few epochs per frame, paced by wall-clock time, and repaints the loss curve as it goes. There is no durationInFrames to know. There is no frame 212 to seek to. There is no Remotion <Player> anywhere on the page.

So when I say "add a transport bar to the live lessons so the whole product feels like one thing," I'm saying something harder than it sounds. The transport bar was built on the frame clock. Its scrubber called seekTo. Its progress came from onFrameUpdate. Its whole notion of "where are we" was a frame number. Hand it a live lesson and every one of those assumptions is a null reference.

The wrong fixes

There are two tempting ways to handle this, and both are traps.

The first is to leave live mode alone — no bar, no scrubber, no progress, the way it already was. This is the path of least resistance and it's why the live instruments shipped bare for months. The cost is invisible until you list it: no replay (an embedded MNIST demo just sat there, classified once, done), and no video export, because export was wired straight into the timeline player. A whole class of lessons quietly missing two features the rest of the site had, for no reason a learner could see.

The second trap is worse: rebuild the bar for live mode. Copy the JSX, swap the Remotion calls for the live view's own play/pause, ship it. Now you have two transport bars that look identical and drift the moment anyone touches one. I've done this kind of thing before and paid for it every time — the second copy is where the bug you fixed in the first copy comes back to life.

The actual fix is to notice that the transport bar never needed a frame clock. It needed four verbs and a status.

Name the verbs, let each engine answer

Strip the bar down to what it actually asks of whatever is behind it, and the list is short. It needs to be able to say play, pause, reset, and "give me your canvas" (for export). And it needs to read back a little status: are you playing, how far along are you, are you at the end, and what's the one-line label for right now.

That was enough of a contract to write down:

interface LivePlaybackHandle {
  play(): void;
  pause(): void;
  reset(): void;
  getCanvas(): HTMLCanvasElement | null;
}

interface LivePlaybackStatus {
  playing: boolean;
  progress: number;   // 0→1, however the view wants to define "progress"
  atEnd: boolean;
  stepLabel: string;
}

The training loop satisfies this contract by exposing its rAF loop: play resumes the loop, progress is epochs / cap, atEnd is "converged or hit the cap." MNIST satisfies it by driving its auto-draw: play replays the seeded stroke, progress is how far the pen has gotten, getCanvas hands back the thing it's been painting. Neither knows the bar exists. They register a handle, push a status on each render, and go back to their own clocks.

The bar, for its part, stopped caring what's behind it. The scrubber became a read-only progress display (a live run has no frame to seek to, so dragging it is meaningless and I made it inert rather than lie about it). The play button calls handle.play(). That was enough coupling.

And the Remotion player? It already is this contract, basically — play, pause, seekTo(0), and the canvas is right there in the DOM. So timeline and operation lessons keep their frame clock and their scrubbing, the live lessons drive themselves, and one bar sits on top of all three. The dependency got inverted: the bar used to depend on Remotion, and now Remotion is just one of several things that can answer the bar's questions.

The unification is the behavior, not the widget

Here's the part that's easy to miss, and the part that bit me.

Putting the same bar on every lesson is cosmetic. The thing that actually makes the product feel like one product is that the bar behaves the same everywhere — and behavior doesn't come for free just because the pixels match. You have to state the rules, or every engine reinvents them slightly wrong.

The behavior timeline had already taught me, long before any of this: a run opens paused, Play runs the seeded demo, and when the run is spent it freezes at "done." From the frozen "done" state, Play means replay — start over. It does not mean resume, because there's nothing to resume; the run is over.

When I wired up the training loop, I got this subtly wrong, and it was a useful reminder that a shared semantic has to be written down. The training loop has several ways to finish: it can converge, it can hit the epoch cap, or it can diverge if the learning rate is too hot. I made play replay after a cap or a divergence — but after convergence, I let it resume, quietly training past the point it had declared itself done. It seemed harmless. It was the exact drift the whole exercise was supposed to kill: one terminal state out of three behaving differently from its siblings, for no reason a user could model. I caught it, folded all three terminal states into the same "Play means replay" behavior, and the live bar started feeling like the timeline bar instead of merely looking like it.

The same pattern kept showing up. A consistent look across three engines is a stylesheet. A consistent feel is behavior you wrote down and checked against every engine, including the one that's a hand-rolled training loop with opinions of its own.

The payoff was a feature I didn't rebuild

The reason any of this was worth doing shows up in video export.

Recording a lesson to MP4 used to be timeline-only, because the recorder reached straight into the Remotion player, grabbed its canvas, and drove the frame clock from 0 to the end while a MediaRecorder captured each frame. Every assumption in that sentence is a Remotion assumption.

But look at what recording actually needs: a canvas to capture, and something that will play a run from start to finish. I already named both of those. getCanvas() is in the contract. "Drive a run 0→end" is just reset() then play() and wait for atEnd. So the recorder became a shared hook that takes a canvas source and lets the player drive the run — and the instant it did, video export lit up for all three modes at once. You can now export an MP4 of a sort, of a stack demo, or of the neural net drawing and second-guessing a digit you trained it to misread — same button, gated by a single canRecordVideo flag instead of a playerMode === "timeline" check.

I didn't build live-mode export as a separate feature. Once the verbs were named, export turned out to already work in terms of them.

That's the shape I ended up trusting. When I find a UI welded to one engine's clock now, the useful question is what the UI is actually demanding — usually a handful of verbs and a status — and whether each engine can answer in its own way. The play button gets simpler, not smarter. It just stops needing to know what it's playing.

See it both ways: watch the neural net draw its own digit with the same transport bar bubble sort uses — then hit record and export the run, a button that until recently only one of these lessons had.