ClawHub

TokenFlow 词元流

Data & APIs

Convert files and URLs to structured text using the TokenFlow API with customizable per-filetype conversion and fallback options.

Install

openclaw skills install tokenflow

TokenFlow

Name: tokenflow
Version: 2.0.0
Description: Convert files and URLs to structured text using the TokenFlow API
Author: Iron Lion International
License: MIT

Overview

This skill enables agents to automatically convert supported file types and URLs to structured text using the TokenFlow API. Configuration happens agent-side — edit config.json to set per-filetype behavior.

Supported Platforms: OpenClaw, Hermes

Requirements

  • TOKENFLOW_API_KEY — Your TokenFlow API key (required)
  • Internet access to reach https://tokenflow.fly.dev

Installation

OpenClaw

openclaw skills install tokenflow

Or manually copy this skill directory to ~/.openclaw/skills/.

Hermes

Copy the tokenflow/ skill directory to ~/.hermes/skills/.

Configuration

Edit config.json in the skill directory:

{
  "apiKey": "your-api-key-here",
  "apiUrl": "https://tokenflow.fly.dev",
  "filetypeBehavior": {
    "docx":   { "action": "convert", "askEachTime": false },
    "xlsx":   { "action": "convert", "askEachTime": true },
    "pdf":    { "action": "convert", "askEachTime": false },
    "audio":  { "action": "convert", "askEachTime": false }
  },
  "outputFormat": "markdown",
  "maxRetries": 1,
  "fallbackBehavior": "use_file"
}

Filetype Behavior

FiletypeExtensionsactionaskEachTime
docx.docx, .pptx"convert" or "skip"true = prompt user each time
xlsx.xlsx, .xls, .csv"convert" or "skip"true = prompt user each time
pdf.pdf"convert" or "skip"true = prompt user each time

| audio | .wav, .mp3, .ogg, .flac, .m4a | "convert" or "skip" | true = prompt user each time |

Options

  • outputFormat: "markdown" (default) or "html". Agents should use "markdown".
  • maxRetries: 0–3. Retry count on failure.
  • fallbackBehavior: "use_file" (use original file on failure) or "fail" (report error)

Supported File Types

  • Documents: .pdf, .docx, .pptx, .html, .htm
  • Spreadsheets: .xlsx, .xls, .csv
  • Data: .json, .xml, .txt, .md
  • Audio: .wav, .mp3, .ogg, .flac, .m4a → transcription
  • Archives: .zip

Agent Behavior

On File Attachment

1. Check if extension is supported
   └─ No → Skip, use file as-is

2. Look up filetypeBehavior in config
   ├─ action = "skip" → Skip
   ├─ askEachTime = true and not explicit → Return shouldAsk signal
   └─ action = "convert" → Proceed

3. For XLSX: call /convert/preview to get sheet names
   └─ If multiple sheets, pass sheets="all"

4. Call TokenFlow API: POST /convert
   Headers: Authorization: Bearer {apiKey}
   Body: multipart/form-data with file + output_format=markdown

5. If success (200):
   └─ Return structured text to agent

6. If failure or empty result:
   ├─ Retries < maxRetries → Retry (step 4)
   ├─ Fallback = use_file → Use original file, log warning
   └─ Fallback = fail → Report error to user

On URL Paste

  1. Detect URL in user message
  2. Call POST /convert/url with {url: "..."}
  3. Return structured text from the webpage

API Integration

POST /convert

curl -X POST "https://tokenflow.fly.dev/convert" \
  -H "Authorization: Bearer ${TOKENFLOW_API_KEY}" \
  -F "file=@document.pdf" \
  -F "output_format=markdown"

Response: JSON with markdown or html field.

POST /convert/preview (XLSX only)

curl -X POST "https://tokenflow.fly.dev/convert/preview" \
  -H "Authorization: Bearer ${TOKENFLOW_API_KEY}" \
  -F "file=@data.xlsx"

Response: {"sheets": ["Sheet1", "Sheet2"]}

GET /health

curl "https://tokenflow.fly.dev/health"

GET /usage

curl -H "Authorization: Bearer ${TOKENFLOW_API_KEY}" \
  "https://tokenflow.fly.dev/usage"

POST /convert/url

curl -X POST "https://tokenflow.fly.dev/convert/url" \
  -H "Authorization: Bearer ${TOKENFLOW_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com"}'

Error Handling

ScenarioBehavior
API key invalidLog error, use original file (if fallback=use_file)
Quota exceeded (429)Report to user: "TokenFlow quota exceeded"
Rate limited (429)Wait 1s, retry once if retries > 0
File too large (413)Skip conversion, use original file
Unsupported format (400)Skip conversion, use original file
API timeout (5s)Fail fast, use original file
Empty resultTreat as failure, apply fallback

Changelog

v2.0.0 (2026-05-13)

  • Agent-agnostic: supports OpenClaw and Hermes
  • Configuration moved agent-side (config.json)
  • Per-filetype behavior with askEachTime option
  • PDFs return markdown with base64-embedded images when outputFormat=markdown
  • Standalone image conversion removed (base64 always inflates tokens)

v1.2.0 (2026-05-05)

  • Added persistent config file
  • Added automatic first-run onboarding
  • Added AGENTS.md sync protocol

v1.0.0 (2026-04-27)

  • Initial release

Support

More in Data & APIs