bytedance-visual-recognition

Multimodal visual recognition via Doubao-Seed (6 models) + Zhipu GLM (2 free models). Image/video to text/code, auto-fallback, batch directory processing, follow-up conversations, local media caching (Temp/), persistent history (vision_history.json), auto IAM console sync. First-run privacy notice, plaintext config.json, cross-platform.

ByteDance Visual Recognition — Doubao-Seed + GLM

Doubao-Seed (6 models) + Zhipu GLM (2 free models). First run auto-generates config.json, fill in one API Key to start. IAM console usage syncs automatically on each recognition.

Privacy & Data Notice

• Network: Selected images/videos and prompts are base64-encoded and sent to Volcengine (Doubao) or Zhipu (GLM) cloud APIs.
• Local cache: Media files temporarily copied to Temp/YYYYMMDD/, default 7-day retention (temp_retention_days in config.json, range 1-3650).
• History: Recognition history stored in vision_history.json, follow-up context in .last_response, auto-cleaned after 7 days. GLM follow-up reuses full message history including base64 media data; Doubao follow-up uses previous_response_id without re-transmitting files.
• Credentials: API Keys stored in plaintext config.json. Do not use personal keys on shared machines.
• IAM sync: Automatically syncs token usage from Volcengine IAM console on each recognition when IAM credentials are configured.
• First run: A privacy notice is displayed once. Continuing past it acknowledges data handling practices.

Setup (pick one)

Doubao (Volcengine)

1. Join the Collaboration Rewards Program for free quota, then get your API Key
2. Create inference endpoints, pick from:

config key	model	priority
doubao_seed_21p_id	Doubao-Seed-2.1-Pro	primary
doubao_seed_21t_id	Doubao-Seed-2.1-Turbo	secondary
doubao_seed_20p_id	Doubao-Seed-2.0-Pro	tertiary
doubao_seed_20c_id	Doubao-Seed-2.0-Code	code-first
doubao_seed_20l_id	Doubao-Seed-2.0-Lite	fallback
doubao_seed_20m_id	Doubao-Seed-2.0-Mini	low-cost

3. Edit config.json, replace "" with actual endpoint IDs.

GLM (Zhipu, free)

1. https://open.bigmodel.cn → get API Key
2. Edit config.json: "zhipu_api_key": "your-key"

Provider filter

provider_mode in config.json:

• 0 = all (default)
• 1 = Zhipu only
• 2 = Doubao only

Test

bash

python doubao_vision_recognize.py --help
python doubao_vision_recognize.py status

---

Commands

command	purpose	example
rec <file> --image|--video --text|--code	recognize	rec a.jpg --image --text
rec <dir> --batch --image|--video --text|--code	batch	rec ./img/ --batch --image --text
ask --text|--code --prompt "..."	follow-up	ask --text -p "details"
status	usage stats
sync	manual console sync
history	7-day history

Parameters

param	desc
--image	image input
--video	video input
--text	text output
--code	code output
--prompt / -p	extra instruction
--batch	directory batch

---

Behavior Rules

1. Trigger only on listed patterns

Activate only when the user's message matches one of the trigger_patterns above. Do NOT activate on loosely related text. If uncertain, ask before executing.

2. Parameter inference

• "recognize/analyze image" → --image --text
• "recognize/analyze video" → --video --text
• "convert to code / UI to code / design to code" → --code
• extra requirements → --prompt "..."
• unsure → ask once if image or video

3. Credential safety

API keys are stored in plaintext config.json. Warn users not to use high-value keys on shared machines. First-run privacy notice already discloses plaintext storage.

---

Limits

• Doubao: 180W tokens per model per day, auto-fallback
• GLM: free, auto-retry on failure (4.6V: 10 retries, 4.1V: 5 retries)
• Image ≤ 15MB, Video ≤ 50MB