ComfyUI Video

Automate AI video generation with ComfyUI and LTX-2.3. Supports text-to-video (T2V), image-to-video (I2V), batch scene rendering for music videos, and multi-...

Security Scan

The skill's files and runtime instructions are coherent with a ComfyUI video-generation helper: it automates browser-side workflow loading and provides SSH-based monitoring tips; it does not request unrelated credentials, external downloads, or hidden endpoints.

Purpose & Capability

The name/description match the contents: guidance for ComfyUI + LTX-2.3, a browser-side automation script, workflow node mappings, and SSH-based monitoring. The large-model and GPU requirements are appropriate for the stated task. There are no unrelated credentials, binaries, or external services requested.

Instruction Scope

SKILL.md stays on-topic: it instructs how to load workflows, tune nodes, run batch scenes using the included browser JS helper, and how to check progress via SSH on a host running ComfyUI. It does not instruct reading unrelated local/system files or sending data to unknown endpoints. It does assume you have SSH access to the ComfyUI host and filesystem.

Install Mechanism

No install spec; this is instruction-only with a small browser script. No downloads or archive extraction are requested. The included script runs in the ComfyUI web UI context and contains no obfuscated code or external network calls.

Credentials

The skill does not declare any required environment variables or credentials (metadata shows none), which is consistent with the browser-console + SSH usage model. However, the runtime guidance expects SSH access and local model files in /workspace/ComfyUI; users will need appropriate SSH keys/credentials and filesystem access to the remote host — these are not provided by the skill and must be supplied by the user.

Persistence & Privilege

The skill is not force-included (always: false) and does not request persistent privileges or modify other skills. It only exposes a helper on window.comfyui_batch when run in browser context, which is normal for a client-side utility.

Assessment

This skill is internally consistent for automating ComfyUI workflows, but take these precautions before using it: 1) Inspect and run the scripts only in a trusted browser session — avoid pasting unknown JS into consoles on machines you don't control. 2) The skill expects SSH access to a host running ComfyUI and large local model files (paths like /workspace/ComfyUI); make sure you trust that remote host and have the necessary credentials/permissions. 3) No secrets are embedded in the skill, but using it requires you to supply SSH credentials yourself; do not expose keys or reuse credentials you cannot trust. 4) Verify model licensing and content policy compliance for any images/people used. If you want a stricter risk posture, test on an isolated VM or a non-production host first.

ComfyUI Video Generation

Automate AI video generation using ComfyUI + LTX-2.3 model. Ideal for music video (MV) production, multi-scene batch rendering, and AI video content creation.

Requirements

Item	Spec
GPU	≥24GB VRAM (Turing/Ampere/Ada)
ComfyUI	0.17+
PyTorch	2.6+cu124
Access	SSH tunnel forwarding port 18188

Model Setup

Model	Size	Path
LTX-2.3 dev (bf16)	43GB	models/checkpoints/ltx-2.3-22b-dev.safetensors
Gemma 3 12B	23GB	models/text_encoders/comfy_gemma_3_12B_it.safetensors
Distilled LoRA	7.1GB	models/loras/ltxv/ltx2/ltx-2.3-22b-distilled-lora-384.safetensors
Video VAE (bf16)	-	models/vae/LTX23_video_vae_bf16.safetensors

Turing GPUs (e.g., Quadro RTX 8000) do NOT support fp8_e4m3fn. Use bf16/fp16 models only.

Performance Baseline

Per-step time: ~221s (constant, regardless of frame count!)
15 steps: ~57 min
25 steps: ~1h45m
Frames: 72=3s, 121=5s, 480=20s (24fps)

Key insight: Frame count does NOT affect total time. Bottleneck is model forward pass.

Workflow Node Reference

Node	ID	Purpose
LoadImage	2004	I2V reference input
CLIPTextEncode (positive)	2483	Positive prompt
CLIPTextEncode (negative)	2612	Negative prompt
EmptyLTXVLatentVideo	3059	Empty latent
LTXVScheduler	4966	Steps/length params
LoraLoaderModelOnly	4922+	LoRA loader
SaveVideo	4823/4852	Output mp4

Quick Start

Generate a Single Video (I2V)

1. Load workflow: /workspace/ComfyUI/custom_nodes/ComfyUI-LTXVideo/example_workflows/2.3/LTX-2.3_T2V_I2V_Single_Stage_Distilled_Full.json
2. Set params using scripts/batch_scenes.js
3. Click Run
4. Wait ~1 hour
5. Download from /workspace/ComfyUI/output/

Batch Scene Generation

Use scripts/batch_scenes.js for automation:

// Load script first, then configure each scene:
await comfyui_batch.configureScene({
  name: "scene_01",
  prompt: "A lonely girl running through rain at night, neon reflections",
  image: "unified_ref.png",
  steps: 15,
  frames: 72
});
// Click Run, repeat for next scene

Step Count Guide

Steps	Quality	Time/Scene	Use Case
8	Rough	~30min	Quick preview
15	Good	~57min	Recommended sweet spot
25	Best	~1h45m	Final quality output

I2V + LoRA at 15 steps achieves ~90% of 25-step quality with 40% less time.

Troubleshooting

VAEDecode Validation Failed

Error: Exception when validating node: 'VAEDecode' Cause: VAE load timing or insufficient VRAM Fix: Reload the entire workflow (fetch + loadGraphData), wait for models to fully load, then run. Never reload during execution.

Browser Tab Lost

Cause: SSH tunnel disconnected Fix:

1. Rebuild tunnel: ssh -f -N -L 18188:localhost:18188 user@host -p port
2. Navigate to ComfyUI
3. Reload workflow

Inconsistent Characters Across Scenes

Cause: Different reference images per scene Fix: Use the SAME reference image for all scenes. Extract a clear frame from an existing video if needed. The I2V input image dictates the visual baseline.

Output Video Not Saved

Check: ssh -p PORT root@HOST "ls -lht /workspace/ComfyUI/output/*.mp4" Fix: Check for VAEDecode errors in log, then re-run.

Monitoring Progress

# Current sampling progress
ssh -p PORT root@HOST "grep 'it/s' /tmp/comfy.log | tail -1"

# Completion check
ssh -p PORT root@HOST "grep 'Prompt executed' /tmp/comfy.log | tail -1"

# Output files
ssh -p PORT root@HOST "ls -lht /workspace/ComfyUI/output/*.mp4"

Best Practices

1. 15 steps is the sweet spot — I2V converges at 15-20 steps, 25 has diminishing returns
2. Unified reference image — Same input image for all scenes ensures character consistency
3. Reload workflow every time — Avoids VAEDecode validation failures
4. Never reload during execution — Current run will fail
5. Frame selection — 72 frames (3s) for testing, 480 frames (20s) for final output
6. VRAM management — Wait for each generation to complete before starting next

T2V vs I2V Comparison

Mode	Steps	Quality	Notes
T2V (no LoRA)	15	❌ Very blurry	Not recommended
I2V + LoRA	25	✅ Excellent	Major quality improvement
I2V + LoRA	15	✅ Very good	Best time/quality ratio

Conclusion: I2V + LoRA is the recommended combination.

Resources

• scripts/batch_scenes.js — Batch scene automation
• references/workflow_nodes.md — Full node ID mapping
• references/tips.md — Prompt tips, VRAM management, optimization