Download a foreign-language movie/video (or take a local file), transcribe and translate it to English with WhisperX, and recreate the video with English subtitles. Use on requests like "get English subs for this", "translate this movie", "subtitle this YouTube film". Fully local: yt-dlp → WhisperX → ffmpeg.
(or uv pip install whisperx if you use uv.) A venv install commonly isn't left on PATH,
so point the script at it with WHISPERX_BIN (see below) if command -v whisperx doesn't
find it — e.g. WHISPERX_BIN=~/.venvs/whisperx/bin/whisperx.
GPU is optional. The script auto-detects nvidia-smi: if present (Linux/Windows with an
NVIDIA GPU) it uses WhisperX's CUDA default (float16); otherwise it falls back to
--compute_type float32 for CPU (including Apple silicon) inference. CPU-only transcription
is slower but works fine — expect roughly ⅓–½ of the video's runtime (see below).
If any of whisperx/yt-dlp/ffmpeg aren't on PATH, set WHISPERX_BIN, YTDLP_BIN, or
FFMPEG_BIN to their full paths before running the script.
--input — URL (anything yt-dlp handles) or a local file path.
--lang — source language code (tr, pt, es, ja, fr, ...). Always pass it;
check the video first (yt-dlp --skip-download --print "%(title)s") if unsure.
Output lands in the current directory as <name>.subbed.mp4 + <name>.subbed.eng.srt
(srt basename matches the video so VLC auto-pairs the sidecar; override with -o/--outdir
and --name).
Default is a fast soft-mux (subs as a selectable track, no re-encode — Plex/players
handle it fine). Pass --burn only if subs must be in the pixels (slow full re-encode).
--no-translate keeps subs in the original language. --model small trades accuracy for speed.
Known limitation: no speaker labels (diarization)
This skill doesn't label who's speaking. We tried bolting on whisperx --diarize (pyannote) and
it didn't hold up in practice, so it was left out. If you want to add it, here's what to know
going in:
Why a naive bolt-on doesn't work: in translate mode, Whisper emits long (20-30s) subtitle
cues, since alignment isn't possible on translated text. That means a single [SPEAKER_NN]
label ends up stamped across an entire multi-person exchange — wrong, and a wall of text on
screen. The speaker clusterer can also over-split a small cast into extra spurious IDs,
especially with music or overlapping dialogue in the mix.
A design that would work better: (1) transcribe in the original language with alignment
enabled, giving short, per-word-timed cues with clean speaker turns; (2) translate each cue to
English separately (an LLM works well here) while preserving timing and speaker labels — this
step can also infer real character names from context instead of generic SPEAKER_NN IDs.
Passing --min_speakers/--max_speakers to pyannote helps when you know the cast size.
Diarization is the most compute-hungry step in the whole pipeline (roughly +50% runtime on
CPU), so a GPU helps a lot if you go this route. It also needs pyannote installed alongside
whisperx and an HF_TOKEN environment variable with access to pyannote's gated models on
Hugging Face.
Run it in the background — WhisperX on CPU takes roughly ⅓–½ of the video's runtime
(a 2h movie ≈ 30–60 min; faster with a GPU). It prints [movie-subtitler] DONE: <path> at the end.
Gotchas
Whisper only translates into English, not between arbitrary language pairs.
When translating, the script passes --no_align: WhisperX's per-language alignment models
can't align translated English text to foreign audio, so we keep plain Whisper timestamps.
They're accurate to ~1s, fine for subtitles.
If the download 403s or fails, yt-dlp is probably stale: upgrade it (pip install -U yt-dlp
or brew upgrade yt-dlp, depending on how you installed it).
