Alibabacloud Bailian Video Creator

AI-powered video creation and editing supporting text-to-video, image-to-video, reference-to-video, video editing, and video content analysis via Alibaba Clo...

AI Video Creator

Professional-grade AI video creation skill supporting text-to-video, image-to-video, video editing, and video understanding. Built on Alibaba Cloud DashScope API.

Prompt Design Guide

The core formula for high-quality video prompts: [Scene Overview] + [Shot Design] + [Camera Movement] + [Lighting & Atmosphere] + [Visual Style]

Key elements: Camera language (shot size, movement, angle), shot rhythm design, lighting and atmosphere control, visual style reference. For multi-shot videos, use the shot list format Shot N [start-end time] Shot description.

Negative prompt template: blurry, low quality, distorted, glitchy, unnatural movement, jumpy cuts, inconsistent lighting, artifacts, watermark, text overlay, static, frozen frames, uncanny valley

For detailed camera language reference tables, shot list templates, lighting styles, and complete prompt examples, refer to references/prompt-guide.md

Version

v2.1.0 (2026-03-10)

Feature List

Feature	Model	Status	Description
Video Understanding	qwen3.5-plus	✅	Analyze video content with configurable frame extraction rate
Text-to-Video (Multi-shot)	wan2.6-t2v	✅	Generate multi-shot videos from text descriptions
Image-to-Video	wan2.6-i2v	✅	Generate video from image + audio
Reference-to-Video	wan2.6-r2v-flash	✅	Generate multi-character video from multiple reference materials (video/image)
Video Editing (Repainting)	wanx2.1-vace-plus	✅	Video repainting while preserving original motion/structure
Video Region Edit	wanx2.1-vace-plus	✅	Fine-grained editing of specific video regions

Note: The DashScope qwen3.5-plus model's image_url type only supports static image analysis and does not support direct video file upload. To analyze video, extract key frames first and use the image analysis feature.

Prerequisites

Before executing any task, the following checks must be completed:

1. Verify API Key availability: Run from api_key import get_api_key; api_key = get_api_key(). If get_api_key() raises an exception (including "quota exceeded", AuthenticationError, InvalidApiKey, 401, etc.), you must immediately stop execution and display the error message to the user -- do not skip API calls, do not continue with subsequent steps, do not fabricate execution results
2. Confirm remote API availability: This skill completes all video generation and content analysis tasks via the DashScope remote API, without relying on local GPU, PyTorch, Stable Diffusion, etc. As long as the API Key is valid, the skill has full capabilities. Do not claim inability to complete tasks just because local tools are missing
3. API Key security: Never write the full API Key in plain text in script files, logs, or terminal output. Must read via the api_key.py module or environment variable os.environ.get("DASHSCOPE_API_KEY"). Display in masked form when debugging (e.g., sk-***xxx)

Orchestration Logic

Products and Services Involved

All APIs in this skill are called through Alibaba Cloud DashScope (Bailian); no other cloud products are involved.

Component	Purpose
scripts/api_key.py	Unified API Key management (reads from ~/.aliyun/config.json or environment variable)
DashScope Video Generation API	Text-to-video, image-to-video, reference-to-video, video editing (async)
DashScope MultiModalConversation API	Video understanding (sync)
FFmpeg (local tool)	Video format conversion, key frame extraction, and other pre/post-processing (optional)

Feature Selection Decision Tree

Select the corresponding feature based on the user's input materials and intent:

text

User Request
│
├─ User has an existing video to analyze/understand?
│   └─ YES → Video Understanding (video_understanding.py, qwen3.5-plus)
│
├─ User has an existing video to modify?
│   ├─ Modify a local region (has mask)?
│   │   └─ YES → Video Region Edit (video_local_edit.py, wanx2.1-vace-plus)
│   └─ Overall style repainting?
│       └─ YES → Video Editing (video_edit.py, wanx2.1-vace-plus)
│
├─ User has multiple reference materials (character videos/images) to synthesize?
│   └─ YES → Reference-to-Video (reference_to_video.py, wan2.6-r2v-flash)
│
├─ User has a single image to generate video from?
│   └─ YES → Image-to-Video (image_to_video.py, wan2.6-i2v)
│
└─ User only has a text description?
    └─ YES → Text-to-Video (text_to_video.py, wan2.6-t2v)

Quick Reference Table

User Input	Feature	Script	Model
Video URL + analysis question	Video Understanding	video_understanding.py	qwen3.5-plus
Pure text description	Text-to-Video	text_to_video.py	wan2.6-t2v
Image URL (± audio URL)	Image-to-Video	image_to_video.py	wan2.6-i2v
Multiple reference materials (video/image mix)	Reference-to-Video	reference_to_video.py	wan2.6-r2v-flash
Video URL + new style description	Video Editing	video_edit.py	wanx2.1-vace-plus
Video URL + mask image + edit description	Video Region Edit	video_local_edit.py	wanx2.1-vace-plus

Invocation Flow

You must directly run existing scripts in the scripts/ directory to complete tasks. Creating new scripts from scratch as replacements is forbidden. Each feature has a corresponding executable script (see Quick Reference Table). Simply pass the correct parameters and run. The scripts internally encapsulate the complete flow of API Key retrieval, API calls, polling, and result output.

All features share a unified invocation pattern, divided into two categories:

Synchronous call (Video Understanding):

python

from dashscope import MultiModalConversation
response = MultiModalConversation.call(model="qwen3.5-plus", messages=[...])
# Returns analysis result directly

Asynchronous call (All video generation/editing):

Step 1 -- Create task (obtain task_id):

python

response = requests.post(
    "https://dashscope.aliyuncs.com/api/v1/services/aigc/video-generation/video-synthesis",
    headers={"Authorization": f"Bearer {api_key}", "X-DashScope-Async": "enable"},
    json={"model": "wan2.6-t2v", "input": {"prompt": prompt}, "parameters": {...}}
)
task_id = response.json()["output"]["task_id"]

Step 2 -- Poll task status (until SUCCEEDED or FAILED):

python

import time
for _ in range(60):  # Poll up to 60 times, 10-second intervals
    result = requests.get(
        f"https://dashscope.aliyuncs.com/api/v1/tasks/{task_id}",
        headers={"Authorization": f"Bearer {api_key}"}
    ).json()
    status = result["output"]["task_status"]
    if status == "SUCCEEDED":
        video_url = result["output"]["video_url"]
        break
    elif status == "FAILED":
        raise RuntimeError(result["output"].get("message", "Task failed"))
    time.sleep(10)

Step 3 -- Verify result: Confirm that a valid video_url or analysis result has been obtained. Do not end execution prematurely before the task completes.

Key Orchestration Rules

1. API Key retrieval must come first: All scripts call get_api_key() at module load time. If no valid Key is found, an error is raised immediately without making API requests
2. Async tasks require polling: Video generation typically takes 1-5 minutes; you must poll via poll_result() and wait for completion
3. Prompt optimization: For text-to-video and image-to-video, setting prompt_extend=True is recommended to let the model auto-rewrite prompts; for video editing/region edit, set to False to avoid rewrites deviating from original intent
4. Video Editing vs Region Edit selection:
   • User provided a mask image → Use Region Edit
   • User only described a new style without mask → Use Video Editing (Repainting)
5. control_condition selection (Video Editing):
   • Need to preserve facial expressions + body movements → posebodyface
   • Only preserve body movements, change face → posebody
   • Preserve composition and motion trajectories → depth (default, most versatile)
   • Preserve line art structure → scribble

Constraints

All tasks must be completed via the DashScope remote API. Using local tools (OpenCV, PyTorch, Stable Diffusion, FFmpeg analysis, etc.) to replace DashScope API for video generation or content analysis is forbidden. Using third-party alternative services (such as Pollinations.ai, RunwayML, Pika, Luma, etc.) to replace DashScope API is forbidden. Claiming "no video generation capability" or "missing tools" is forbidden -- as long as the API Key is valid, this skill has full remote video generation and understanding capabilities.

1. API Key pre-check: Before any API call, you must verify the API Key is configured via get_api_key(). If get_api_key() fails (including "quota exceeded", 401, InvalidApiKey, etc.), you must immediately stop and report the error -- do not skip API calls, do not fabricate subsequent results
2. Correct model names (no other names may be used):

Feature	Required Model	Incorrect Examples (Forbidden)
Text-to-Video	wan2.6-t2v	wanx2.6-t2v, wanx2.1-t2v-plus, wanx-video-generation-v1
Image-to-Video (HTTP API)	wan2.6-i2v	-
Image-to-Video (SDK)	wan2.6-i2v-flash	-
Reference-to-Video	wan2.6-r2v-flash	-
Video Editing/Region Edit	wanx2.1-vace-plus	-
Video Understanding	qwen3.5-plus	qwen-vl-plus, qwen-vl-max-latest

3. Async tasks must be polled to completion: After creating a task, you must poll task_status until SUCCEEDED or FAILED. Do not end prematurely before the task completes. If task creation fails (e.g., 401 error), you must report the error clearly and prompt the user to check API Key -- do not silently terminate

4. Final result must be verified: Confirm that a valid video_url (video generation) or analysis text (video understanding) has been obtained. Do not return empty results. If the API call failed or was not actually executed, you must honestly report the error. Fabricating analysis results or fictitious video URLs is forbidden

5. API Key security: Never write the full API Key in plain text in generated scripts, logs, or terminal commands. Specific rules:

   ❌ Forbidden (none of the following are allowed):

   python

   api_key = "sk-abc123..."                          # Hardcoding forbidden
   headers = {"Authorization": "Bearer sk-abc123..."}  # Plain text forbidden
   

   bash

   export DASHSCOPE_API_KEY="sk-abc123..."            # Plain text export in scripts forbidden
   

   ✅ Correct (must use the following methods):

   python

   from api_key import get_api_key
   api_key = get_api_key()                            # Securely obtained via module
   # or
   api_key = os.environ.get("DASHSCOPE_API_KEY")      # Obtained via environment variable
   

6. No fabricating results: If the DashScope API call was not actually executed or execution failed, you must clearly inform the user of the failure reason. Do not fabricate video analysis text, fictitious video URLs, or claim task completion when api.json is empty. Writing a script containing API call code ≠ actually executing the API call. The standard: only when the script is actually run and the DashScope API (e.g., MultiModalConversation.call or dashscope.aliyuncs.com/api/v1/...) returns a real result can you claim task completion. API Key management calls (ListWorkspaces, CreateApiKey) do not count as task completion

7. Third-party alternative services forbidden: All video generation and understanding tasks must be completed via DashScope API. Calling Pollinations.ai, RunwayML, Pika, Luma, Kling, or other third-party services as alternatives is forbidden

8. Output file verification: Before claiming task completion, you must confirm that actual output files have been written to disk (e.g., video_analysis_result.txt, downloaded video files, etc.). If the file does not exist or is empty, you must report failure

9. Must use existing scripts and actually execute them: Use existing scripts in the scripts/ directory (e.g., python scripts/video_understanding.py). Creating new scripts from scratch as replacements is forbidden. Scripts must be actually executed (python scripts/xxx.py), and real stdout/stderr output must be captured as the basis for results. If script execution encounters errors, you must display the real error message

Environment Setup

1. Install FFmpeg (Video Processing Tool)

FFmpeg is used for video format conversion, key frame extraction, video trimming/concatenation, and other local processing.

bash

# macOS (Homebrew)
brew install ffmpeg

# Ubuntu/Debian
sudo apt update && sudo apt install ffmpeg

# Windows (Chocolatey)
choco install ffmpeg

# Windows (Scoop)
scoop install ffmpeg

Verify installation:

bash

ffmpeg -version

2. Install Python Dependencies

bash

pip install -r scripts/requirements.txt

Package	Version Constraint	Purpose
dashscope	>=1.25.0,<2.0.0	SDK-based image-to-video, video understanding
requests	>=2.20.0,<3.0.0	HTTP API-based video generation/editing

3. Configure API Key

API Key is managed through the unified scripts/api_key.py module with three-tier retrieval logic:

1. Read ~/.aliyun/config.json current profile's dashscope.api_key → return if found
2. Read environment variable DASHSCOPE_API_KEY → return if found
3. Detect Alibaba Cloud CLI available → auto-call generate_api_key() to create Key and save to config → return
4. All fail → raise error (with CLI installation instructions)

python

# All scripts use this unified approach
from api_key import get_api_key
api_key = get_api_key()

Manual environment variable configuration:

bash

export DASHSCOPE_API_KEY=sk-xxx

Item	Description
Key format	sk-xxx (standard DashScope API Key)
Not usable	sk-sp-xxx (Coding Plan exclusive Key, does not support video generation)
Obtain at	https://bailian.console.aliyun.com/cn-beijing/?tab=app#/api-key

4. Alibaba Cloud CLI Configuration (API Key Auto-Create/Delete)

This skill's scripts/api_key.py uses aliyun modelstudio commands to automatically create and delete API Keys. Complete the following configuration before use:

1. Enable AI-Mode and update plugins

bash

# Enable AI-Mode (allow Agent to call CLI)
aliyun configure ai-mode enable

# Set User-Agent
aliyun configure ai-mode set-user-agent --user-agent "AlibabaCloud-Agent-Skills/alibabacloud-bailian-video-creator"

# Update plugins to latest version
aliyun plugin update

2. Install ModelStudio plugin (if not already installed)

bash

aliyun plugin install --names aliyun-cli-modelstudio --enable-pre

3. Disable AI-Mode after task completion

bash

aliyun configure ai-mode disable

CLI commands involved:

Command	Purpose	Call Location
aliyun modelstudio list-workspaces	Get Bailian Workspace ID	api_key.py: _get_workspace_id()
aliyun modelstudio create-api-key	Create DashScope API Key	api_key.py: generate_api_key()
aliyun modelstudio delete-api-key	Delete cloud API Key	api_key.py: delete_api_key()

Directory Structure

text

alibabacloud-bailian-video-creator/
├── scripts/           # Example code (7 core features + API Key management)
│   ├── requirements.txt
│   ├── api_key.py
│   ├── video_understanding.py
│   ├── text_to_video.py
│   ├── image_to_video.py
│   ├── image_to_video_sdk.py
│   ├── reference_to_video.py
│   ├── video_edit.py
│   └── video_local_edit.py
├── references/        # Reference materials
│   ├── prompt-guide.md
│   ├── api-docs.md
│   ├── models.md
│   └── error-codes.md
└── SKILL.md

Notes:

• scripts/ - Contains only official example code, one file per feature
• references/ - Prompt guide, API documentation, model list, error codes, and other reference materials

Usage Examples

Run any script directly to see detailed parameter descriptions (python scripts/<script_name>.py).

Feature	Script	Key Parameters
Video Understanding	video_understanding.py	video_url, prompt, fps
Text-to-Video	text_to_video.py	prompt (with shot list), duration, size
Image-to-Video (HTTP)	image_to_video.py	prompt, img_url, audio_url, resolution
Image-to-Video (SDK)	image_to_video_sdk.py	prompt, img_url, audio_url, duration
Reference-to-Video	reference_to_video.py	prompt (with Character references), reference_urls
Video Editing	video_edit.py	prompt, video_url, control_condition
Video Region Edit	video_local_edit.py	prompt, video_url, mask_image_url, mask_type

Region URLs

API URLs differ by region. Choose based on your actual needs:

Region	URL
Beijing	https://dashscope.aliyuncs.com/api/v1
Virginia	https://dashscope-us.aliyuncs.com/api/v1
Singapore	https://dashscope-intl.aliyuncs.com/api/v1

Billing

Video Generation Billing

Billed by the duration (seconds) of successfully generated videos. Failed requests are not billed.

Text-to-Video (wan2.6-t2v)

Resolution	Unit Price
720P	¥0.6/sec
1080P	¥1.0/sec

Billing examples:

• Generate 10-second 720P video → ¥6.00
• Generate 10-second 1080P video → ¥10.00

Image-to-Video / Reference-to-Video (wan2.6-i2v-flash / wan2.6-r2v-flash)

Resolution	Video Type	Unit Price
720P	With audio	¥0.3/sec
720P	Silent	¥0.15/sec
1080P	With audio	¥0.5/sec
1080P	Silent	¥0.25/sec

Note: Reference-to-video is billed by input + output total duration. Input video is billed for a maximum of 5 seconds.

Video Editing (wanx2.1-vace-plus)

Resolution	Unit Price
720P	¥0.72/sec

View billing: https://usercenter2.aliyun.com/finance/expense-center/overview

Free Quota

New users receive upon activating Bailian:

• 50 seconds of free video quota
• Valid for: 90 days after activation

References

• Text-to-Video API Documentation
• Image-to-Video API Documentation
• Video Editing API Documentation
• Model List