code2animation

Produce complete code-based animated videos by scripting, generating narration, creating visual assets, and rendering final MP4s using the code2animation fra...

Security Scan

The code and README generally match a code-driven video rendering tool, but the dev-server API spawns shell commands with unsanitized input and the build config can bake an API key into the client — these inconsistencies create a real risk of remote command execution and secret leakage.

Purpose & Capability

The project files (render scripts, Puppeteer, FFmpeg, msedge-tts, TTS metadata handling) align with the stated purpose of code-driven animation and rendering. However there are dependencies and files not documented in SKILL.md (e.g., '@google/genai', 'better-sqlite3' in package.json/pnpm-lock) that are not explained by the description and may indicate unused or extra capabilities.

Instruction Scope

The vite plugin exposes HTTP endpoints (/api/projects and /api/generate-audio) that accept POST data and then execute a local shell command that runs scripts/generate-audio.ts. The command string is composed as `npx tsx "${scriptPath}" ${projectId}` using projectId from the request body without sanitization or argument escaping, enabling potential shell injection if the dev server is reachable. The README and SKILL.md claim the commands are predefined and not user-controllable, which contradicts the server code that accepts external input and uses it in a shell command. Also the package's dev script sets Vite --host=0.0.0.0 and --port=3000 (in package.json), which makes these endpoints reachable from the network if the developer runs the dev server as-is.

Install Mechanism

There is no external install spec (instruction-only install). Dependencies are typical for a Node + Puppeteer + React project; all packages are from normal registries and a pnpm lockfile is included. No remote archive downloads or extract steps were observed.

Credentials

The skill declares no required env vars, but vite.config.ts reads and injects env.GEMINI_API_KEY into the client bundle via define({ 'process.env.GEMINI_API_KEY': JSON.stringify(env.GEMINI_API_KEY) }). That will bake any present GEMINI_API_KEY into the shipped frontend code, potentially leaking secrets to anyone who can load the app. The SKILL.md does not document or justify exposing a GEMINI_API_KEY. Optional env vars mentioned in SKILL.md (PUPPETEER_EXECUTABLE_PATH) are reasonable, but the implicit GEMINI_API_KEY handling is disproportionate and risky.

Persistence & Privilege

The skill does not request always:true and does not alter other skills. The dev server plugin registers local HTTP endpoints and invokes local scripts and child processes (Puppeteer/FFmpeg). Those behaviors are expected for a renderer but gain broader impact if the dev server is bound to 0.0.0.0 or if the agent runs the server with network exposure. Autonomous invocation is allowed by default but does not by itself change this assessment.

Scan Findings in Context

[child_process.exec] expected: Executing render and audio-generation scripts is expected for this project, but the code constructs a shell command using untrusted input (projectId) and runs it via execAsync, which can lead to command injection if the dev server is reachable remotely.

[exposed_env_define] unexpected: vite.config.ts injects GEMINI_API_KEY into the client bundle via define(). Embedding a secret into frontend code is not appropriate and risks leaking the key to anyone who can access the app.

What to consider before installing

This package mostly does what it says (creates and renders code-driven animations), but it has two important red flags you should address before running it in a networked environment: (1) The Vite dev plugin exposes /api/generate-audio which runs a shell command built directly from the incoming projectId. If you run the dev server bound to 0.0.0.0 (package.json dev script currently does), a remote attacker could POST a malicious projectId and execute arbitrary commands. Mitigations: run the dev server only on localhost, change the dev script to --host=127.0.0.1, or remove/secure the API route with authentication and argument escaping (use spawn with arg arrays or a safe argument-escaping function). (2) vite.config.ts will bake any GEMINI_API_KEY environment variable into the client bundle, exposing it to users — do not set sensitive API keys in the environment when building/running this app, or remove that define() usage and keep keys server-side. Additional suggestions: review scripts/generate-audio.ts and scripts/render.ts to confirm they do not themselves call unsanitized shell commands or perform network calls to unexpected endpoints; prefer child_process.spawn with argument arrays instead of exec to avoid shell interpolation; avoid running the dev server as root or exposing it publicly; and if you need remote triggering, add authentication and strict input validation (whitelist project IDs). If you want, I can point out exact lines to change to fix the command-injection and key-exposure issues.

AgentSaaS Video Editor Skill

A comprehensive video editing and rendering skill that enables AI agents to create code-driven animations with text-to-speech narration and smooth transitions.

Purpose

This skill allows agents to:

• Create and preview interactive video projects with animations
• Generate TTS audio narration using Microsoft Edge TTS
• Render complete videos with synchronized audio and visual effects
• Support both portrait and landscape video formats
• Apply smooth transition effects between media elements

Core Capabilities

1. Interactive Video Preview

• Real-time preview of video projects in the browser
• Playback controls for testing and debugging
• Support for transitions, media clips, and timing adjustments
• Frame-by-frame seeking for precise editing
• Live transition preview with easing effects

2. Transition System

• transitionIn: Each media defines its own entrance animation
• Supported transitions: fade, zoom, slide2Left, slideUp, none
• Easing: Built-in easeOutCubic for smooth slide and zoom animations
• stayInClip: Media can persist throughout entire clip duration
• Cross-clip transitions: Automatic handling of clip boundaries

3. TTS Audio Generation

• Automated text-to-speech using Microsoft Edge TTS (msedge-tts)
• Support for multiple voices (English and Chinese)
• Word-level timing metadata for lip-sync and animations
• Audio file caching for faster previews

4. Video Rendering

• Automated frame-by-frame rendering using Puppeteer
• FFmpeg integration for video encoding and audio mixing
• 30 FPS output at 1920x1080 (landscape) or 1080x1920 (portrait)
• Deterministic rendering for consistent results
• Transition effects preserved in final output

Project Configuration Format

Media Item Properties

• src: HTML filename in the footage directory
• words: Trigger phrase from speech that activates this media
• transitionIn: Entrance animation type (optional)
• transitionDuration: Duration in seconds (optional, default: 0.6s)
• stayInClip: If true, media remains visible until clip ends (optional)

Transition Types

• fade: Opacity transition (0 → 1)
• zoom: Scale transition (2x → 1x) with opacity
• slide2Left: Horizontal slide from right (100% → 0%)
• slideUp: Vertical slide from bottom (100% → 0%)
• none: No transition effect

Transition Behavior

• transitionIn: Defines how this media enters the scene
• transitionDuration: Duration in seconds (default: 0.6s)
• stayInClip: If true, media remains visible until clip ends
• Easing: slide2Left and slideUp use easeOutCubic for smooth deceleration

Reference Implementation

See public/projects/agentSaasPromoVideo.json for a complete working example demonstrating all transition types and stayInClip behavior.

Technical Requirements

System Dependencies

• Node.js: 18 or higher
• FFmpeg: Required for video encoding and audio mixing
• Chromium/Chrome: Used by Puppeteer for headless rendering

Node.js Dependencies

• React & Vite: Frontend framework and build tool
• Puppeteer: Headless browser for frame capture
• msedge-tts: Microsoft Edge TTS for audio generation
• Express: Optional HTTP server (for TTS API endpoint)
• Motion (Framer Motion): Animation library
• Tailwind CSS: Styling framework

Shell Commands Used

This skill executes the following shell commands:

Audio Generation

npx tsx scripts/generate-audio.ts <projectId>

• Reads project JSON configuration
• Generates MP3 audio files using Edge TTS
• Saves word-level timing metadata

Video Rendering

node scripts/render.js <projectId> [--portrait]

• Starts a local Vite dev server
• Launches Puppeteer to capture frames
• Uses FFmpeg to encode video and mix audio
• Cleans up temporary files

FFmpeg Operations

• Frame encoding: ffmpeg -framerate 30 -i frames/frame-%05d.jpg -c:v libx264 ...
• Audio mixing: ffmpeg -i video.mp4 -i audio1.mp3 -i audio2.mp3 -filter_complex ...

Browser Detection

• Uses which command to find Chrome/Chromium on Linux/macOS
• Respects PUPPETEER_EXECUTABLE_PATH environment variable

API Endpoints (Optional)

The skill may expose an HTTP endpoint for TTS generation:

POST /api/tts
Content-Type: application/json

{
  "text": "Text to speak",
  "voice": "en-US-GuyNeural",
  "rate": "+0%",
  "pitch": "+0Hz"
}

This endpoint is optional and only used when pre-generated audio files are not available.

Security Considerations

File System Access

• Reads from: public/projects/<projectId>/
• Writes to: public/projects/<projectId>/audio/, public/video/
• Creates temporary directories for frame storage
• Cleans up temporary files after rendering

Network Access

• Starts local HTTP server on port 5175+ (configurable)
• Connects to Microsoft Edge TTS service (external)
• No external API keys required for basic functionality

Process Execution

• Spawns child processes for: Vite dev server, FFmpeg encoding
• Uses execSync for: browser detection, audio generation trigger
• All commands are predefined and not user-controllable

Environment Variables

Optional configuration:

• PUPPETEER_EXECUTABLE_PATH: Custom browser path for Puppeteer
• FASTMCP_LOG_LEVEL: Logging level (default: ERROR)

Project Structure

public/
  projects/
    <projectId>/
      <projectId>.json       # Project configuration
      footage/               # HTML/CSS media components
      audio/                 # Generated TTS audio files
        0.mp3, 1.mp3, ...
        0.json, 1.json, ...  # Word timing metadata
  video/
    render-<projectId>-landscape.mp4
    render-<projectId>-portrait.mp4

Usage Example

# 1. Generate audio for a project
pnpm generate-audio agentSaasPromoVideo

# 2. Preview in browser
pnpm dev

# 3. Render final video
pnpm render agentSaasPromoVideo

# 4. Render portrait version
pnpm render agentSaasPromoVideo --portrait

HTML Animation Guidelines

When creating HTML animations for video rendering, use the CSS variable timeline model.

Core Model

• Renderer controls time: Puppeteer sets --t every frame.
• Page only renders state: DOM = f(t).
• No lifecycle animation APIs (play/start/reset) and no hidden runtime state.
• Transition system handles entrance effects: Don't implement slide/fade transitions in HTML - use the project's transitionIn property instead.

✅ Required Patterns

• Define timeline root:

  :root { --t: 0; }
  

• Every animated property must derive from --t.
• Always clamp normalized progress values:

  --p: clamp(0, calc((var(--t) - var(--start)) / var(--duration)), 1);
  

• Express initial/ending states directly in CSS (seek-safe at any frame).
• Use small deterministic JS only for content mapping (e.g., subtitle/text index from t).
• Let transition system handle entrance: Focus on content animation, not entrance effects.

🚫 Forbidden Patterns

• transition
• animation / @keyframes
• window.registerFrameAnimation(...)
• requestAnimationFrame loops for timeline progression
• Implicit time from Date.now() / performance.now() for visual state
• Manual entrance transitions: Don't implement slide/fade in HTML - use transitionIn in project config
• Fade-out effects: Elements should not disappear after animation completes. Use opacity: var(--p) instead of opacity: calc(var(--p) * (1 - var(--fade))) to keep elements visible at their final state.

Recommended Template

.element {
  --start: 0.5;
  --duration: 1;
  --p: clamp(0, calc((var(--t) - var(--start)) / var(--duration)), 1);

  opacity: var(--p);
  transform: translateY(calc((1 - var(--p)) * 20px));
}

Easing (without transition)

Use math on progress directly:

--p: clamp(0, calc((var(--t) - var(--start)) / var(--duration)), 1);
--ease-out: calc(1 - (1 - var(--p)) * (1 - var(--p)));
opacity: var(--ease-out);

JS Hook Pattern (text/content only)

<script>
  const labels = ['A', 'B', 'C'];
  const el = document.getElementById('label');

  window.onTimelineUpdate = (t) => {
    const idx = Math.floor(Math.max(0, t) / 1.2) % labels.length;
    el.textContent = labels[idx];
  };
</script>

Time Semantics (t vs globalTime)

• onTimelineUpdate(t, globalTime) supports two time domains:
  • t: clip-local time (resets to 0 when clip changes). This is the default for most HTML animations.
  • globalTime: continuous timeline across clips. Use only when an element must stay continuous through cross-clip transitions.
• Do not assume t is media-local. If a media appears mid-clip, t may already be large when it first becomes visible.
• For media-local behavior (e.g., toggle starts animating when this media appears), anchor from first visible globalTime and derive:
  • local = globalTime - mediaStartGlobalTime
• Keep fallback for compatibility:

  window.onTimelineUpdate = (t, globalTime) => {
    const g = Number.isFinite(globalTime) ? globalTime : t;
    // use `t` for normal clip-local animation, `g` only when continuity is required
  };
  

Determinism Checklist

• Seeking to any t yields exactly one deterministic frame.
• Animation state must not depend on "previous frame".
• Cross-clip transition visuals should be continuous in both clips.
• Final frame (t = totalDuration) must remain on the last clip (no wrap to first clip).

Limitations

• Requires FFmpeg to be installed on the system
• TTS generation requires internet connection (Microsoft Edge TTS)
• Rendering is CPU-intensive and may take several minutes
• Maximum TTS text length: ~1000 characters per clip
• Frame capture requires sufficient disk space

Transparency Statement

This skill executes shell commands and spawns child processes for video rendering. All operations are limited to:

1. Starting a local development server (Vite)
2. Running FFmpeg for video encoding
3. Launching Puppeteer for frame capture
4. Detecting browser executables on the system

No arbitrary code execution or user input is passed to shell commands. All file paths and commands are predefined and validated.