Dub Anything – open-source AI dubbing with voice cloning and bilingual subs

[swax]

There are movies I want to watch that have never been translated into English. I built this tool so I can generate subtitles and dubs that are good enough to watch.

Samples

Animation dubbed into English
Live action dubbed into English

The samples use cloned versions of the original actors’ voices, using their original delivery as a reference. There’s also an American English voice option.

Instructions

To use it, give your preferred AI coding agent (Claude Code, Codex, OpenCode, etc.) the repo link below and the video you want to dub, and it should guide you through the process.

Repo: https://github.com/swax/dub-anything

How it works

The agent breaks down the dialogue, figures out who is saying what, corrects the transcript, and translates each line to fit the original timing. It then generates the dub, checks the generated speech through transcription, and retries lines that fail those checks. It can also translate on-screen text and display it as captions.

The output keeps the original video without re-encoding, adds the dubbed audio alongside the original audio, and includes subtitles in both languages. Music and sound effects are separated from the dialogue and mixed back underneath the dub. It runs in half the time if you just want subtitles and no dub.

Requirements and runtime

The software is free and open source, and the speech models run locally on Windows with an AMD or NVIDIA GPU (6 GB VRAM minimum) or macOS. Your coding agent may have its own costs and processes transcripts and inspected frames wherever its model runs.

A full movie can take many hours. You still need to listen to the result and judge the voices and acting, but I’ve found it useful for watching things I otherwise couldn’t understand.

Interested to hear what people think of the samples.