video-translation

Translate and dub videos from one language to another, replacing the original audio with TTS while keeping the video intact.

Security Scan

The skill appears to do what it claims (subtitle translation + TTS dubbing + audio replacement) and its included scripts are local and straightforward, but there are a few metadata mismatches and privacy-relevant instructions (uploading original audio to a third‑party TTS backend and persisting an API key) that you should be aware of before installing.

Purpose & Capability

The skill's code and SKILL.md match the stated purpose (download subtitles, translate them, generate TTS audio, and replace the video's audio). However, the registry metadata lists no required env vars or binaries while SKILL.md explicitly requires NOIZ_API_KEY and ffmpeg — this mismatch is a documentation/metadata inconsistency that should be corrected.

Instruction Scope

Runtime instructions direct the agent to send the original video's audio (via --ref-audio-track) to the Noiz TTS backend so it can clone the voice and align emotion/duration. That means user audio will be uploaded to a third-party service; the SKILL.md also tells the agent to optionally persist the NOIZ_API_KEY into a project .env or run the tts.sh config helper. These behaviors are coherent with the skill's purpose but raise privacy and secret-handling concerns and should be made explicit to end users.

Install Mechanism

This is an instruction-only skill with two small local scripts. No network downloads or archive extraction are performed by the skill itself, and the scripts perform only local file operations and ffmpeg invocations. That is low-risk from an install mechanism perspective.

Credentials

The SKILL.md requires a NOIZ_API_KEY (for the Noiz TTS backend) and ffmpeg. These are proportionate to a TTS/voice-cloning workflow, but the registry metadata omitted these requirements. Also note the SKILL.md instructs the agent to offer to persist the API key to .env or via tts.sh — storing secrets in plaintext project files has risks and should be optional and documented.

Persistence & Privilege

The skill does not request always:true, does not modify other skills, and does not require system-wide privileges. It does suggest optionally writing the NOIZ_API_KEY to a project .env, which affects repository files but is within expected installer behavior for an API-backed skill.

Assessment

Before installing: 1) Be aware this skill uploads original audio to the Noiz backend when you pass --ref-audio-track (voice cloning) — do not upload sensitive or private recordings unless you trust Noiz and have permission. 2) The SKILL.md requires NOIZ_API_KEY and ffmpeg, but the registry metadata omitted these; ensure you obtain a NOIZ_API_KEY and have ffmpeg installed. 3) The agent may offer to persist the API key to a .env file or call tts.sh config — prefer not to store secrets in plaintext in shared repos; consider using a secure secret store. 4) Verify the referenced youtube-downloader and tts skills come from trusted sources before cloning them into your skills/ folder. 5) If you need higher assurance, inspect the tts skill's code (network calls, where it sends audio) and test the workflow with non-sensitive sample videos first.

Translate a video's speech into another language, using TTS to generate the dubbed audio and replacing the original audio track.

Triggers

• translate this video
• dub this video to English
• 把视频从 X 语译成 Y 语
• 视频翻译

Use Cases

• The user wants to watch a foreign language YouTube video but prefers to hear it in their native language.
• The user provides a video link and explicitly requests changing the audio language.

Workflow

When the user asks to translate a video:

1. Download Video & Subtitles: Use the youtube-downloader skill to download the video and its subtitles as SRT. Make sure you specify the source language to fetch the correct subtitle.

   python path/to/youtube-downloader/scripts/download_video.py "VIDEO_URL" --subtitles --sub-lang <source_lang_code> -o /tmp/video-translation
   

2. Translate Subtitles: Read the downloaded .srt file. Translate its contents sentence by sentence into the target language using the following fixed prompt. Keep the exact same SRT index and timestamp format!

   Translation Prompt:

   Translate the following subtitle text from <Source Language> to <Target Language>. Provide ONLY the translated text. Do not explain, do not add notes, do not add index numbers. The translation must be colloquial, natural-sounding, and suitable for video dubbing.

   Save the translated text into a new file translated.srt.

3. Generate Dubbed Audio: Use the tts skill to render the timeline-accurate audio from the translated SRT. The Noiz backend automatically aligns the duration of each sentence to the original video's subtitle timestamps.

   To ensure the cloned voice matches the original speaker's exact tone and emotion for each sentence, pass the original video file to --ref-audio-track. The TTS engine will automatically slice the original audio at each subtitle's exact timestamp and use it as the reference for that specific segment.

   Create a basic voice_map.json:

   {
     "default": {
       "target_lang": "<target_lang_code>"
     }
   }
   

   Render the timeline-accurate audio:

   bash skills/tts/scripts/tts.sh render --srt translated.srt --voice-map voice_map.json --backend noiz --auto-emotion --ref-audio-track original_video.mp4 -o dubbed.wav
   

4. Replace Audio in Video: Use the replace_audio.sh script to merge the original video with the new dubbed audio. To keep the original video's non-speech audio background outside of translated segments, pass the --srt file.

   bash skills/video-translation/scripts/replace_audio.sh --video original_video.mp4 --audio dubbed.wav --output final_video.mp4 --srt translated.srt
   

5. Present the Result: Return the final_video.mp4 file path to the user.

Inputs

• Required inputs:
  • VIDEO_URL: The URL of the video to translate.
  • target_language: The language to translate the audio to.
• Optional inputs:
  • source_language: The language of the original video (if not auto-detected or specified).
  • reference_audio: Specific audio file/URL to use for voice cloning instead of the dynamic original video track.

Outputs

• Success: Path to the final video file with replaced audio.
• Failure: Clear error message specifying whether download, TTS, or audio replacement failed.

Requirements

• Dependencies (other skills)
  • youtube-downloader (crazynomad/skills) — SKILL.md
    Install: clone or copy the skills/youtube-downloader directory from crazynomad/skills into your skills/ folder so that skills/youtube-downloader/scripts/download_video.py is available.
  • tts (NoizAI/skills) — SKILL.md
    If not already in this repo: clone or copy the skills/tts directory from NoizAI/skills into your skills/ folder. Ensure skills/tts/scripts/tts.sh and related scripts are present.
• NOIZ_API_KEY configured for the Noiz backend. If it is not set, first guide the user to get an API key from https://developers.noiz.ai/api-keys. After the user provides the key, ask whether they want to persist it; if they agree, either write/update NOIZ_API_KEY=... in the project's .env file or run bash skills/tts/scripts/tts.sh config --set-api-key YOUR_KEY to store it.
• ffmpeg installed.

Limitations

• The source video must have subtitles (or auto-generated subtitles) available on the platform for the source language.
• Very long videos may take a significant amount of time to translate and dub.