Universal Video to S3 Uploader

Download videos from YouTube, Twitter/X, TikTok, Douyin, Bilibili and upload to S3-compatible storage. Universal video downloader with smart quality selectio...

Security Scan

The skill's code mostly matches its description (download videos and upload to S3) but contains multiple inconsistencies and risky behaviors (hardcoded workspace paths, mismatched config locations, self-modifying script, and use of shell exec) that warrant caution before installing.

Purpose & Capability

The declared purpose (download videos and upload to S3) aligns with the code's use of yt-dlp, ffmpeg and the AWS S3 SDK. However there are notable mismatches: the SKILL.md/README instructs the user to place credentials in ~/.youtube-s3-uploader.yml, while the runtime scripts expect a different config file (/home/node/.openclaw/workspace/.r2-upload.yml). Hardcoded workspace paths for yt-dlp, ffmpeg, downloads and config indicate this package was written to run inside a specific agent/container environment rather than as a general CLI, which is not explained in the documentation.

Instruction Scope

Runtime scripts call external binaries via execSync/exec (yt-dlp, ffmpeg, curl) and read/write files under hardcoded paths in /home/node/.openclaw/workspace. SKILL.md claims 'no external APIs' and instructs a user-local config but the code will look for and require a YAML config in the workspace path. The scripts will read local files, execute shell commands, and can run curl to verify URLs — behaviors consistent with a downloader but broader than the user-facing config/instructions imply and potentially surprising to non-technical users.

Install Mechanism

There is no install spec in the registry metadata (instruction-only), but the package contains Node code and package.json with dependencies (@aws-sdk/client-s3, js-yaml). Nothing in the manifest downloads arbitrary code at runtime from untrusted URLs; however there are no explicit install-time steps documented for installing Node deps or ensuring yt-dlp/ffmpeg are available at the hardcoded paths. The absence of an install recipe plus hardcoded binary locations is fragile and risky operationally.

Credentials

The skill declares no required environment variables, yet it requires S3 credentials — stored in a YAML config — to function. The documentation tells users to put credentials in ~/.youtube-s3-uploader.yml, but the runtime code reads /home/node/.openclaw/workspace/.r2-upload.yml (and other scripts default to that path). This mismatch could cause credential misplacement or accidental use of credentials in unintended locations. The package also contains a script (scripts/add-import.js) that programmatically edits another script, which increases the surface where secrets/config could be read or changed unexpectedly.

Persistence & Privilege

The skill is not marked always:true and does not request elevated platform privileges. However it includes a helper (scripts/add-import.js) that modifies code files, and other scripts write/read files under hardcoded workspace directories. These behaviors create persistence of data (downloads, config with secrets) on disk and alter local files — normal for a CLI uploader but something to be aware of before running in a sensitive environment.

What to consider before installing

This package appears to implement a video downloader → S3 uploader, but it has several red flags you should consider before installing or running it: - Do not trust the README alone: the runtime scripts expect a config at /home/node/.openclaw/workspace/.r2-upload.yml, while docs tell you to use ~/.youtube-s3-uploader.yml. Verify which config file the code will actually read so you don't accidentally leak credentials or leave them in an unexpected location. - Inspect the code locally before running. The scripts use execSync/exec to run yt-dlp, ffmpeg and curl, and they read/write files under hardcoded workspace directories. Make sure those paths are acceptable and won't expose any sensitive data on your machine or agent workspace. - Be cautious about credentials: only provide S3 keys you control and consider using a scoped, low-privilege key or a dedicated test account. Do not reuse high-privilege AWS keys. - The package contains a script that edits other script files (scripts/add-import.js). Consider removing or reviewing that behavior if you plan to run the tool. - Ensure required binaries exist at the expected locations or modify the scripts to point to safe, known paths. Alternatively, run this in an isolated environment (container or VM) where accidental exposure is contained. If you want to proceed safely, run the code in a sandbox, correct the config path to the one you control, and test with temporary, limited-permission S3 credentials.

Universal Video to S3 Uploader v3.0.1

A universal video downloader that supports multiple platforms and uploads to S3-compatible storage.

🚀 Version 3.0.1 - Filename Optimization

What's New:

• ✅ Clean Filenames: Removes all punctuation marks, emojis, and special characters from filenames
• ✅ Better Unicode Support: Proper handling of Chinese punctuation and emojis
• ✅ Platform Detection: Automatic detection of video platform (YouTube, Twitter/X, TikTok, etc.)
• ✅ Universal Support: Works with 1000+ websites via yt-dlp

Breaking Changes:

• ❌ NOT backward compatible: v3.x is a complete rewrite from v2.x
• ❌ New API: Universal video-to-s3-universal.js script
• ❌ Platform Detection: Automatically detects video platform

Why v3.0.1?

v3.0.1 improves filename handling:

1. Clean filenames - Removes all punctuation and emojis
2. Better S3 compatibility - Avoids issues with special characters
3. Improved readability - Clean, readable filenames without clutter

What's New:

• ✅ S3 Multipart Upload for All Files: v2.0.0 uses S3 multipart upload for all file sizes
• ✅ Memory Optimization: Chunked reading to avoid memory overflow
• ✅ Error Recovery: Automatic retry mechanism for failed uploads
• ✅ Smart Chunk Sizing: Automatically adjusts chunk size based on file size
• ✅ Performance Improvements: Better progress tracking and speed calculation
• ✅ Fixed Help System: Proper help information display

Breaking Changes:

• ❌ NOT backward compatible: v2.0.0 is a complete rewrite
• ❌ Old scripts removed: All v1.x scripts have been replaced
• ❌ New API: Uses S3 multipart upload for all files, not just large ones

Why v2.0.0?

v2.0.0 fixes critical issues in v1.x:

1. Fixed help system - Proper --help display
2. Consistent API - Always uses multipart upload
3. Better error handling - Improved error messages
4. Cleaner codebase - Removed compatibility layers

Features

• One-command workflow: Download YouTube video → Upload to S3 → Get access URL
• S3-compatible: Works with Cloudflare R2, AWS S3, MinIO, and any S3-compatible storage
• Automatic cleanup: Removes local files after successful upload
• Progress tracking: Real-time upload progress and speed calculation
• Secure by default: Uses your own S3 credentials, no external dependencies
• Flexible configuration: Multiple bucket support, custom paths, metadata
• Smart upload method: Automatically uses S3 Multipart Upload for large files (>50MB)
• Memory efficient: Chunked reading to avoid memory overflow
• Error recovery: Retry mechanism for failed uploads

Prerequisites

1. S3-compatible storage (Cloudflare R2, AWS S3, MinIO, etc.)
2. S3 credentials (Access Key ID and Secret Access Key)
3. Bucket created with appropriate permissions

Quick Start

1. Install the skill

clawhub install youtube-s3-uploader

2. Configure your S3 storage

Create ~/.youtube-s3-uploader.yml:

# Default bucket to use
default: my-videos

# Bucket configurations
buckets:
  my-videos:
    endpoint: https://your-s3-endpoint.com
    access_key_id: YOUR_ACCESS_KEY_ID
    secret_access_key: YOUR_SECRET_ACCESS_KEY
    bucket_name: my-videos
    region: auto  # Use "auto" for Cloudflare R2, or specific region for AWS S3
    # Optional: Custom public URL (e.g., CDN domain)
    # public_url: https://cdn.yourdomain.com

3. Download and upload a YouTube video

youtube-s3-upload https://youtu.be/VIDEO_ID

Configuration Details

Cloudflare R2 Example

buckets:
  r2-storage:
    endpoint: https://ACCOUNT_ID.r2.cloudflarestorage.com
    access_key_id: YOUR_R2_ACCESS_KEY_ID
    secret_access_key: YOUR_R2_SECRET_ACCESS_KEY
    bucket_name: video-storage
    region: auto

AWS S3 Example

buckets:
  aws-s3:
    endpoint: https://s3.us-east-1.amazonaws.com
    access_key_id: YOUR_AWS_ACCESS_KEY_ID
    secret_access_key: YOUR_AWS_SECRET_ACCESS_KEY
    bucket_name: my-video-bucket
    region: us-east-1

MinIO / Self-hosted Example

buckets:
  minio:
    endpoint: http://localhost:9000
    access_key_id: minioadmin
    secret_access_key: minioadmin
    bucket_name: uploads
    region: us-east-1

Usage

Basic usage

# Download and upload a YouTube video (original method)
youtube-s3-upload https://youtu.be/8uZGlzWA4oo

# Download and upload using FIXED method (recommended for large files)
npm run youtube-to-s3-fixed -- https://youtu.be/8uZGlzWA4oo

# Specify custom S3 path
youtube-s3-upload https://youtu.be/VIDEO_ID --path videos/2026/march/my-video.mp4

# Use specific bucket (from config)
youtube-s3-upload https://youtu.be/VIDEO_ID --bucket backup-bucket

# Keep local file after upload (for debugging)
youtube-s3-upload https://youtu.be/VIDEO_ID --keep-local

Upload existing video file

# Upload a local video file to S3
upload-to-s3 /path/to/video.mp4

# With custom S3 path
upload-to-s3 /path/to/video.mp4 --path archived/videos/special.mp4

Test S3 connection

# Test S3 connection and bucket access
test-s3-connection

# Test specific bucket
test-s3-connection --bucket my-bucket

Output

After successful processing, you'll get:

🎉 Processing Complete!

📊 Results:
Video Title: What Made the Turtle Cry!? Vedal's First Interaction with Neuro 3D
Original Size: 41.22 MB
Upload Time: 11.12 seconds
Average Speed: 3.71 MB/s

🔗 S3 Access URL:
https://s3.yourdomain.com/bucket-name/videos/2026-03-25/video-title.mp4

📁 S3 Path:
bucket-name/videos/2026-03-25/video-title.mp4

💡 Tip: This URL may require authentication. Use presigned URLs for temporary access.

Advanced Features

Presigned URLs

Generate temporary access URLs (default: 1 hour):

generate-presigned-url videos/2026/march/my-video.mp4

# Custom expiration (e.g., 24 hours)
generate-presigned-url videos/2026/march/my-video.mp4 --expires 24h

List uploaded videos

# List recent uploads
list-s3-uploads

# List from specific bucket
list-s3-uploads --bucket my-bucket

# List with details
list-s3-uploads --detailed

Delete files from S3

# Delete a file
delete-from-s3 videos/old-video.mp4

# Delete with confirmation
delete-from-s3 videos/old-video.mp4 --confirm

Environment Variables

• YOUTUBE_S3_CONFIG: Path to config file (default: ~/.youtube-s3-uploader.yml)
• YOUTUBE_S3_DEFAULT_BUCKET: Override default bucket
• YOUTUBE_S3_KEEP_LOCAL: Keep local files (default: false)
• YOUTUBE_S3_DEBUG: Enable debug logging

Security Best Practices

1. Use IAM roles when possible instead of long-term credentials
2. Set short expiration for presigned URLs (minutes, not days)
3. Enable bucket versioning for accidental deletion protection
4. Configure lifecycle rules to automatically delete old files
5. Use bucket policies to restrict access by IP or referrer

Troubleshooting

Common Issues

1. "Invalid credentials": Check your Access Key ID and Secret Access Key
2. "Bucket does not exist": Create the bucket in your S3 provider's dashboard
3. "Access denied": Verify bucket policies and IAM permissions
4. "SSL certificate error": For self-signed certificates, use HTTP or add certificate exception
5. "Upload timeout": Large files may need multipart upload - use the fixed version
6. "Memory overflow": Use the fixed version with chunked reading
7. "Large file upload fails": Use youtube-to-s3-fixed.js for files >50MB

Fixed Version for Large Files

For large video files (>50MB), use the fixed version that implements S3 Multipart Upload:

# Using npm script
npm run youtube-to-s3-fixed -- https://youtu.be/VIDEO_ID

# Direct script execution
node scripts/youtube-to-s3-fixed.js https://youtu.be/VIDEO_ID

# Upload existing large file
node scripts/fixed-upload-video-to-s3.js /path/to/large-video.mp4

The fixed version:

• Uses S3 Multipart Upload for files >50MB
• Implements chunked reading to avoid memory overflow
• Includes retry mechanism for failed uploads
• Shows real-time progress and speed
• Automatically selects best method based on file size

Debug Mode

Enable debug logging to see detailed information:

YOUTUBE_S3_DEBUG=true youtube-s3-upload https://youtu.be/VIDEO_ID

License

MIT-0 - Free to use, modify, and redistribute. No attribution required.

Credits

Created by 西米露 (Simeilu) - A complete YouTube to S3 workflow for personal media storage.