How the engine works

Songs become visuals through a fixed set of stages. The cheap thinking happens first in text; a human direction point sits in the middle; the expensive video generation only ever renders already-approved ideas.

The stages

The seven moves per piece

Every render, whether a comic panel or a video shot, is assembled by one command (engine make SLUG KIND) that makes the same seven moves. The job folder it writes is the only thing that ever leaves this machine.

1. Grab data — the song note: lyrics, the LLM's line-by-line read, motifs, story seed, audio features if ingested.
2. Grab model — an open-weight model chosen from the registry by job kind and tier (draft / standard / premium).
3. Align compute — the runner for that tier: local box, rented pod, or hosted service.
4. Give direction — the shot list and style block assembled from the analysis; the actual prompt the worker renders.
5. Output lands — a job folder (spec.json + results/) is written and queued; the render drops results back into it.
6. Evaluate — a human gate. Luke reviews what came back and keeps, rerolls, or reshapes.
7. Next move — the song advances a stage (engine advance), or the job reruns.

What exists today

The vault, the ideation loop, this monitor, and the whole job loop are live. engine make assembles a real job folder (data, model, compute, direction) for any analysed song; engine work is the render worker — it turns each shot into a ComfyUI graph from a swappable recipe and submits it to a ComfyUI server on the GPU box, landing frames in the job's results with a manifest. An offline mode emits those exact graphs without a GPU, so the loop is inspectable end to end here before any metal is rented. engine advance moves a song through the stages as pieces land.

Still to plug in behind the same job folder: video generation, voice and lip-sync, and the beat-synced assembly — each a new render recipe and, where needed, a new graph builder, never a change to the loop above.

Five lanes, one spine

Lyric videos, comics and graphic novels, avatar-presented course videos, vertical micro-dramas, and whole-album aggregates all share the same early stages, so work done once serves every format.