surreal-sync

Data migration and synchronization to SurrealDB from MongoDB, PostgreSQL, MySQL, Neo4j, Kafka, and JSONL. Full and incremental CDC sync. Part of the surreal-...

Security Scan

The skill's description, examples, and requirements are internally consistent with a CLI data-sync tool; it is instruction-only, asks for no unrelated environment credentials, and does not request elevated persistence — but exercise normal caution around how you install and pass database passwords.

Purpose & Capability

Name/description (SurrealDB sync from various sources) matches the SKILL.md content: CLI usage, supported sources, CDC modes, and examples are exactly what you'd expect for a migration/sync tool.

Instruction Scope

SKILL.md gives concrete CLI examples and connection-string usage only; it does not instruct the agent to read unrelated system files or secret stores. Note: examples show credentials passed as command-line flags (e.g., --surreal-password), which can expose secrets via process listings or logs — this is a usability/security note rather than a scope mismatch.

Install Mechanism

The skill has no install specification and no code files (instruction-only). It suggests installing a Rust binary via 'cargo install surreal-sync', which is a plausible and proportional install step for a Rust CLI; the skill itself will not download or write code to disk automatically.

Credentials

No required environment variables, no unrelated credentials, and CLI examples require only the typical connection strings/credentials for the source DB and SurrealDB destination — all proportional to the stated purpose.

Persistence & Privilege

always is false and the skill does not request persistent system-wide privileges or modification of other skills/configs. Autonomous invocation is allowed (platform default) but not combined with other concerning privileges.

Assessment

This skill appears coherent and focused on data migration to SurrealDB. Before installing or running it: (1) verify the upstream repository (the SKILL.md references github.com/surrealdb/surreal-sync) and prefer official release binaries or pinned commits; (2) avoid putting passwords on the command line where possible — use config files, environment variables, or secret stores to prevent exposure in process lists and logs; (3) run initial syncs in a test environment with limited-permission accounts and review CDC setup steps (wal2json, triggers, Kafka consumer configs) to avoid accidental data or production-impacting operations; (4) because this is instruction-only, the skill will not automatically install code — you or your agent will run the cargo install; confirm you trust the upstream binary source before installation.

Surreal-Sync -- Data Migration and Synchronization

Surreal-Sync is a CLI tool for migrating data from various database sources to SurrealDB with full and incremental synchronization via Change Data Capture (CDC).

Supported Sources

Source	Full Sync	Incremental CDC	Method
MongoDB	Yes	Yes	Change streams
MySQL	Yes	Yes	Trigger-based CDC + sequence checkpoints
PostgreSQL (triggers)	Yes	Yes	Trigger-based CDC + sequence checkpoints
PostgreSQL (wal2json)	Yes	Yes	Logical replication with wal2json plugin
Neo4j	Yes	Yes	Timestamp-based tracking
JSONL Files	Yes	N/A	Batch import from JSON Lines
Apache Kafka	Yes	Yes	Consumer subscriptions with deduplication

Quick Start

# Install surreal-sync (Rust binary)
cargo install surreal-sync

# Full sync from PostgreSQL (trigger-based)
surreal-sync from postgres trigger-full \
  --connection-string "postgresql://user:pass@localhost/mydb" \
  --surreal-endpoint "http://localhost:8000" \
  --surreal-username root \
  --surreal-password root \
  --to-namespace prod \
  --to-database main

# Incremental CDC from PostgreSQL (wal2json)
surreal-sync from postgres wal2json \
  --connection-string "postgresql://user:pass@localhost/mydb" \
  --surreal-endpoint "http://localhost:8000" \
  --surreal-username root \
  --surreal-password root \
  --to-namespace prod \
  --to-database main

# Full sync from MongoDB
surreal-sync from mongo full \
  --connection-string "mongodb://localhost:27017/mydb" \
  --surreal-endpoint "http://localhost:8000" \
  --surreal-username root \
  --surreal-password root \
  --to-namespace prod \
  --to-database main

# Batch import from JSONL
surreal-sync from jsonl import \
  --file data.jsonl \
  --surreal-endpoint "http://localhost:8000" \
  --surreal-username root \
  --surreal-password root \
  --to-namespace prod \
  --to-database main

# Consume from Kafka
surreal-sync from kafka consume \
  --bootstrap-servers "localhost:9092" \
  --topic my-events \
  --surreal-endpoint "http://localhost:8000" \
  --surreal-username root \
  --surreal-password root \
  --to-namespace prod \
  --to-database main

CLI Pattern

surreal-sync from <SOURCE> <COMMAND> \
  --connection-string [CONNECTION STRING] \
  --surreal-endpoint [SURREAL ENDPOINT] \
  --surreal-username [SURREAL USERNAME] \
  --surreal-password [SURREAL PASSWORD] \
  --to-namespace <NS> \
  --to-database <DB>

Key Features

• Automatic schema inference and SurrealDB table creation
• Record ID mapping from source primary keys
• Relationship extraction and graph edge creation
• Configurable batch sizes and parallelism
• Resumable sync with checkpoint tracking
• Deduplication for Kafka consumers

Full Documentation

See the main skill's rule file for complete guidance:

• rules/surreal-sync.md -- source configuration, schema mapping, CDC setup, conflict resolution, and production deployment
• surrealdb/surreal-sync -- upstream repository