Spikecv Helper

Help AI Agents answer questions and execute tasks for SpikeCV, an ultra-high-speed spike camera vision framework. Use when the user asks about spike cameras,...

SpikeCV Agent Skill

last updated: 2026-04-28

This document is designed to equip an AI Agent (or developer) with the necessary prior knowledge and instructions to answer questions and execute tasks within the SpikeCV repository.

Any command execution or conceptual question should be informed by the context in this document and the attached references.

Before reading: If you're unfamiliar with spike cameras or SpikeCV's architecture, first read references/About_SpikeCV.md.

---

🧭 Quick Lookup

Use this table to jump to the right section. If it's not here, the answer isn't in this skill.

You want...	Go to...
What is SpikeCV?	references/About_SpikeCV.md
Install SpikeCV	§ Installation
SpikeCV API docs	spikecv.readthedocs.io
SpikeCV Project directory structure	§ Project Layout
Run spikecv CLI (download, track, reconst)	§ CLI Reference
Full publication list (up-to-date)	spikecv.github.io/publications.html
Algorithm and it's theory	search for docs/source/核心操作.rst 🇨🇳 or source code under SpikeCV/spkProc/ 🇬🇧 SpikeCV/examples 🇬🇧, you might get it wrong, so notify user to look up the details in publication for exact theory
SpikeCV event, competition, Team contact, contribution, hardware	references/About_SpikeCV.md
Dataset descriptions	references/About_SpikeCV.md
Troubleshooting	§ Gotchas & Common Errors

Not in this skill? → Defer to the user or suggest SpikeCV website and SpikeCV team contact ([e-mail]).

---

📦 Installation

Prerequisites:

• Python ≥ 3.10
• Python virtual environment (Conda recommended): see install_miniconda.md for a step-by-step setup guide from scratch.

git clone https://github.com/Zyj061/SpikeCV
cd SpikeCV
pip install .[cli]

Optional extras:

pip install .[tracking]   # for SNNTracker/SpikeSORT

Verify:

spikecv --help

Quick health check (downloads ~40 MB and runs TFSTP reconstruction):

spikecv data download --dataset recVidarReal2019 --agent-used
cd datasets/
spikecv proc reconst --agent-used

---

🛠 CLI Reference

SpikeCV provides the spikecv command for wrapper interactions. After installation, you can use it to download datasets and run processing algorithms.

All commands accept --agent-used to return structured JSON output. Always include this flag.

Note: Feel free to explore other CLI options and parameters as needed. The following examples only showcase partial subcommands and parameters, the CLI may support additional features or algorithms. Always refer to --help for the most up-to-date command options and usage instructions.

spikecv data download

Download a dataset from the OpenI platform:

spikecv data download --dataset <name> --local-dir <path> --agent-used

Dataset	For Task
recVidarReal2019	Reconstruction
motVidarReal2020	Tracking

⚠️ Path gotcha: Data lands in <local-dir>/<dataset>/. Default local-dir is datasets/, so your dat files end up at datasets/recVidarReal2019/classA/car-100kmh.dat, NOT recVidarReal2019/classA/car-100kmh.dat.

spikecv proc reconst

Reconstruct visible images from a raw .dat spike stream. Default algorithm: TFSTP.

# From the datasets/ directory:
cd datasets/
spikecv proc reconst --dat-file-path recVidarReal2019/classA/car-100kmh.dat --agent-used

# Or with an absolute/explicit path:
spikecv proc reconst --dat-file-path /path/to/recVidarReal2019/classA/car-100kmh.dat \
                     --yaml-file-path /path/to/recVidarReal2019/config.yaml \
                     --agent-used

Key parameters:

• --dat-file-path: Path to .dat spike stream file
• --yaml-file-path: Path to config.yaml for the dataset (default: recVidarReal2019/config.yaml)
• --begin-idx: Starting frame index (default: 500)
• --block-len: How many spike frames to process (default: 1500)
• --stp-d, --stp-F, --stp-f: STP model parameters

⚠️ Path gotcha: The default yaml-file-path is recVidarReal2019/config.yaml (relative to cwd). If you download the dataset using the CLI, it lives under datasets/recVidarReal2019/. You must either cd datasets/ first or pass explicit paths.

spikecv proc track

Run multi-object tracking on spike data. Default algorithm: SNNTracker.

spikecv proc track --scene-idx 0 --metrics --agent-used

Key parameters:

• --scene-idx: Scene index 0–6
• --metrics: (Optional) Calculate MOTA/IDF1 metrics (requires ground truth labels)

⚠️ Dataset: Tracking uses motVidarReal2020, not recVidarReal2019.

---

📁 Project Layout

SpikeCV/
├── SpikeCV/
│   ├── cli/               # CLI command definitions (typer)
│   ├── device/            # Hardware driver for spike cameras
│   ├── examples/          # Standalone test scripts (**NOT for agent use**, use CLI instead)
│   ├── metrics/           # Quantitative evaluation (PSNR, SSIM, MOTA, IDF1, AEPE)
│   ├── spkData/           # Data loaders + config.yaml parsing
│   │   └── load_dat.py    # ParaDict generation, SpikeStream class
│   ├── spkProc/           # ⭐ Core algorithms
│   │   ├── filters/       # STP filter (background removal)
│   │   ├── reconstruction/ # TFI, TFP, TFSTP, SSML
│   │   ├── tracking/      # SNNTracker, SpikeSORT
│   │   ├── detection/     # STDP, Motion-based detection
│   │   ├── recognition/   # RPSNet, SVM, VGG
│   │   ├── depth_estimation/ # SpikeT (Transformer)
│   │   ├── optical_flow/  # SCFlow
│   │   └── augment/       # Data augmentation
│   ├── utils/             # Path helpers
│   └── visualization/     # Video generation from spikes
├── docs/
│   ├── spike_algo.md      # deprecated
│   ├── data_processing.md # deprecated
│   ├── tools.md           # deprecated
│   ├── examples.md        # deprecated
│   └── source/            # 🇨🇳 Sphinx sources (readthedocs)
├── Publications.md        # 📄 Publication list (may be stale — use website instead)
├── README.md / README_en.md
└── CONTRIBUTING.md / CONTRIBUTING_en.md

🌐 Documentation by Language

File(s)	Language	Best for
docs/source/*.rst	🇨🇳 Chinese	API, little Algorithm theory, code examples
SpikeCV/spkProc/**/*.py (source code)	🇬🇧 English	Algorithm math + implementation details
README_en.md, CONTRIBUTING_en.md	🇬🇧 English	Setup, contribution workflow
Publications.md	🇬🇧 English	Publication list (stale — see website)

---

🚨 Rules (SHOULD NOT DO)

These are not suggestions — following them prevents errors and hallucinations:

1. Use CLI, not example scripts — If a task has a spikecv CLI command, use it. Do NOT run SpikeCV/examples/test_snntracker.py or similar standalone scripts directly.
2. Always include --agent-used — Otherwise the CLI prints log text, not structured JSON.
3. Do not modify SpikeCV code — Suggest the user contact the team instead ([e-mail]).
4. Do not hallucinate external resources — For contacts, purchases, or contributions, only use info from references/About_SpikeCV.md.
5. For publication/paper queries → always use the website — Publications.md in the repo may be outdated. Point to spikecv.github.io/publications.html as the canonical source.

---

🐛 Gotchas & Common Errors

Symptom	Cause	Fix
"Data path '...config.yaml' does not exist"	--yaml-file-path is relative but you're not in the right directory	cd datasets/ first, or pass full path
spikecv proc track fails	Wrong dataset — using recVidarReal2019 instead of motVidarReal2020	Download correct dataset first
--scene-idx out of range	motVidarReal2020 only has scenes 0–6	Use a value in [0, 6]
--scene-idx in range but still broken	some of the scene data might not have uploaded by the developer, check carefully if the corresponding scene appear in the downloaded dataset	don't use the scene that's not downloaded
spikecv not found	CLI not installed or not in PATH	Run pip install .[cli] from SpikeCV root
--agent-used not working (no JSON output)	Using an older SpikeCV version	Update to latest, or check spikecv --help for flag name