My coding agent now narrates, scores and reviews the video of what it built

In September I open-sourced agentic-media-kit: the coding agent records the app it just built, cuts a Reel and hands me a link I watch on my phone. It worked. But every video came out silent.

Silent is fine for checking a screen. It isn't fine for announcing one. A tutorial without a voice explains half of it, and an ad without music looks unfinished. So the kit got sound. The video below was made entirely by it: voiceover, score, subtitles and cuts, with no video editor, no stock music and no voice actor.

Six modes, and one that picks for you

Not every video wants the same things, so audio became a mode:

Mode	Voice	Music	Spoken subtitles
captions	–	–	–
voice	✓	–	–
voice+captions	✓	–	✓
music+captions	–	✓	–
music+voice	✓	✓	–
music+voice+captions	✓	✓	✓

Each step's text is always on screen, because feeds autoplay muted. The spoken subtitles are an extra track that highlights each word as it's said.

The most useful one in practice is --mode auto. The agent passes the video's goal and the kit picks the mode and says why. An onboarding tutorial gets voice, music and subtitles, because a voice explains better than a headline. A launch ad gets music and text. A testimonial gets voice and subtitles.

node scripts/audio.mjs tour.json tour-voiced.json --name onboarding \
  --mode auto --goal "onboarding tutorial" --tts gemini
# mode: music+voice+captions (goal matches "tutorial")

The script voices each step, composes the score and stretches each scene to fit its line. At the end it writes a props file that Remotion renders as is. The voice can come from Gemini, from ElevenLabs (which returns the exact timing of every word), or from macOS say for offline drafts.

A score composed for each cut

Music is where I changed my mind the most. The first version used a chord synthesized in ffmpeg: license-free, but thin. The generative music APIs cost money, have licenses that change by plan, or, in Lyria's case, simply didn't work on a free key.

The answer came from another open-source project, teardown, which scores its videos with a synthesis engine in plain Python: drums, bass, e-piano, reverb and mastering, standard library only. No model, nothing to compile, no per-track cost. I ported that engine (MIT, credited in the repo) and wrote four moods on top of it: calm, tech, upbeat and warm.

Because the music is composed from the video's own timeline, the kit knows exactly where every beat lands. So the cuts are aligned to it: each scene's length is rounded to a whole number of beats, and each cut gets a subtle accent. The score is mastered to −14 LUFS, the loudness Reels, TikTok and Shorts normalize to, and it ducks on its own under the voice.

And because I don't trust "it should be in sync", there's a checker: check-beats.mjs detects the music's onsets in the final file and measures how far each cut is from the nearest beat. In the kit's example tour, every cut landed within 15 ms of the beat.

A reviewer that never saw the process

This is the idea I like most, and it came from looking at how other projects do it.

The agent that built the video read the brief, chose the copy and watched it render. It sees what it meant, not what's on screen. So the last check goes to a second, fresh agent that gets only three things: a sheet with one frame per scene, the props file and the rules. Not the conversation, not the history.

The rules are numeric and run first, in lint-video.mjs: scenes of at least 1.5 s, captions with enough reading time, narration that fits its scene, a 28 px minimum font size, at most one zoom per two scenes, Reels up to 90 s. The reviewer handles what rules can't: whether the subtitles are legible, whether something got cropped, whether the story serves the goal.

On the first round, it failed the video, and it was right. The subtitle was a grey box covering a quarter of the phone screen, and it looked like a button. It also caught that the brand in the copy didn't match the app on screen. I fixed both. On the second round, it insisted the subtitle over the screen read as part of the app. That's where the rule I wrote for the process itself kicked in: if the same issue survives two rounds, a person decides, not the loop. I kept the subtitles over the screen, which is the TikTok and Reels convention, and recorded the decision in the docs for future reviewers.

What broke along the way

Again, the most useful part was the errors that don't look like errors.

• The audio was 43 ms late. The beat checker failed every cut by nearly the same amount, +42 ms. The cause: the AAC encoder in Remotion's MP4s puts 2048 samples of silence at the start and doesn't mark them in the file. Nobody hears it, but the measurement did. webm-to-mp4.sh now trims that delay before publishing.
• Normalizing silence breaks the encoder. Every Remotion render has an audio track, even a silent one, and ffmpeg's loudness filter turns digital silence into NaN. Normalization now only runs when the track's peak is above −70 dB.
• The pace rule measured the wrong thing. The first version of the lint divided the word count by the length of the generated voice clip. That measures the synthetic voice's speaking speed, which no amount of trimming changes, so the lint said "use fewer words" forever. The right rule compares the words with the time the scene has.
• The subtitle outline came out hollow. -webkit-text-stroke with paint-order should draw the outline behind the letters. In headless Chrome it drew it on top, and the letters looked hollow. The fix was to build the outline from a ring of shadows.
• The voice model read the direction out loud. Gemini 3.x TTS spoke the style prefix along with the text: "Try it today." went from 1.9 s to 4.9 s. The script now sends the delivery direction in a form those models don't read aloud.
• The free quota mixes voices. When one Gemini model hits its quota, the script moves to the next. The video still comes out, but different lines can come from different models, with slightly different timbres. For a final piece, use a paid key or ElevenLabs.

Try it

Still MIT, and it runs with no keys at all: the offline say voice plus the procedural score already make a complete video you can listen to.

git clone https://github.com/rwspatin/agentic-media-kit && cd agentic-media-kit
npm install
node scripts/audio.mjs apps/studio/examples/tour-audio.json /tmp/tour.json \
  --name demo --mode music+voice+captions --tts say
cd apps/studio && npx remotion render Tour-Reel out/tour.mp4 --props=/tmp/tour.json

The repo has the audio guide, the review rules and the code for this launch video: https://github.com/rwspatin/agentic-media-kit.

No agent listens to a video for you. If you try it, tell me what sounded off. That's the kind of feedback no LUFS meter replaces.

• Renan