Visible Text Extractor

Extract and reconstruct as much visible text as possible from webpage URLs, article pages, screenshots, long images, image directories, and GIFs. Use when th...

Security Scan

The skill's code and instructions broadly match an extractor's purpose, but it omits declared dependencies and relies on external tools/scripts and optional remote delivery (Feishu), creating provenance, dependency, and exfiltration concerns the user should understand before running it.

Purpose & Capability

The skill claims to extract visible text from pages/images and the included scripts implement that. However the package metadata lists no required binaries or credentials while scripts clearly call node, ffmpeg, and an external/local OCR stack (paths like /root/.openclaw/.../ocr-local/scripts/ocr.js and /root/.openclaw/venvs/ocrstack/bin/python). The absence of declared runtime requirements is an incoherence: a consumer would legitimately need those dependencies to run the skill.

Instruction Scope

SKILL.md and USAGE.md instruct the agent to download pages/images, render pages via a browser fallback, extract GIF frames, run OCR, and produce docx/JSON/markdown. That scope matches the stated purpose. Some scripts also reference other local skill scripts (feishu sender, ocr-local) and absolute workspace paths and will download arbitrary image URLs discovered in pages — expected for this task but worth noting because it increases the runtime network surface (and may access internal-only URLs if present).

Install Mechanism

There is no install spec (instruction-only with bundled scripts) so nothing is fetched at install time. Runtime, however, depends on external binaries and other skill scripts (node, ffmpeg, local OCR scripts). The lack of an explicit install section or dependency declaration is the main issue, not the install mechanism itself.

Credentials

The skill declares no required env vars/credentials, but several scripts can invoke a Feishu file-sender script and will send a generated docx if a user-supplied --send-feishu-receive-id is passed. The code will also invoke external local tools under absolute paths. Requiring no credentials is coherent for read-only extraction, but the optional remote-send behavior and implicit dependencies on other local skill code are not documented in metadata and increase the risk of unintended data sharing or failure due to missing components.

Persistence & Privilege

always is false and the skill does not request permanent inclusion or modify other skills' configs. It writes temporary files and output artifacts in specified output paths, which is expected for this workflow.

What to consider before installing

This skill appears to do what it says, but before running it you should: (1) confirm required runtime tools exist (node, ffmpeg, Python, the local OCR stack referenced at /root/.openclaw/.../ocr-local and any virtualenv); the package metadata currently does not declare these dependencies, so expect runtime errors otherwise; (2) be aware it downloads arbitrary images/frames referenced by pages and writes temp files — if pages contain internal URLs this could cause server-side requests to your internal network (SSRF-like behavior); (3) the scripts optionally send the produced docx to Feishu when a receive-id is supplied — do not provide a receive-id unless you trust the destination; (4) review or supply trusted implementations for the referenced external scripts (ocr.js, feishu_file_sender.py) since the skill delegates OCR and delivery to them; (5) run the skill in an isolated environment (or sandbox) if you plan to process sensitive pages. If the publisher updates the package metadata to list required binaries and external script dependencies explicitly (and documents the optional Feishu delivery clearly), and if you verify the referenced local scripts are trusted, this assessment could be upgraded to benign.

Use this skill to turn a webpage article, URL, screenshot set, long image set, or local image collection into complete, readable, reusable text.

Core workflow

1. Extract visible body text from the main source.
2. Discover ordered images and GIF-like assets.
3. OCR image content when needed.
4. Preserve a raw/audit layer.
5. Run a human-first cleanup pass.
6. Classify image-like content by likely information type.
7. Reconstruct image content into human-readable supplements instead of raw OCR dumps.
8. Output polished markdown first; keep raw OCR as JSON or appendix data.

What this skill is good at

• General webpage article extraction
• WeChat / 公众号 article extraction with special handling
• News pages, blogs, tutorials, explainers, and image-heavy articles
• Screenshots and long-image OCR
• Image directory OCR in display order
• GIF frame extraction plus OCR when ffmpeg is available
• Rebuilding noisy OCR into a cleaner reading version
• Producing either reader-friendly clean output or full transcript-style output

Main script

• scripts/extract_visible_text.py

Supporting resources

• scripts/postprocess_ocr_text.py — clean OCR output, merge broken spacing, remove obvious garbage, and regroup into readable sections
• scripts/extract_with_browser.js — browser-rendered fallback for JS-heavy pages
• scripts/extract_gif_frames.sh — GIF frame extraction via ffmpeg
• scripts/build_deliverable_docx.js — convert cleaned markdown into a Word document
• scripts/build_transcript_docx.js — convert transcript-style markdown into a Word document
• scripts/build_authorized_capture_docx.py — one-step pipeline for already-authorized browser pages, saved HTML, screenshots, and mixed inputs into clean markdown + JSON + Word deliverable
• scripts/extract_visible_text_deliverable.py — one-step pipeline from source input to clean markdown + JSON + Word deliverable
• scripts/extract_visible_text_transcript_deliverable.py — one-step pipeline for transcript-style full extraction output
• scripts/extract_visible_text_reading_order_deliverable.py — one-step pipeline for reading-order transcript output
• scripts/build_wechat_interleaved_docx.py — reconstruct WeChat article reading order by interleaving extracted body blocks and image OCR text in original flow order
• scripts/ocr_high_accuracy.py — higher-accuracy OCR with preprocessing variants and segmented long-image handling
• references/output-schema.md — target output structure and cleanup rules
• references/deliverable-workflow.md — one-step deliverable workflow guidance
• references/troubleshooting.md — failure patterns, environment limits, and how to respond cleanly
• references/product-positioning.md — what mature deliverable quality means for this skill
• references/generalization-plan.md — how to evolve the skill across travel deals, rule pages, event posters, and tutorial long images
• references/universal-article-extractor-spec.md — generalized capability contract for article, mixed-media, and screenshot-heavy extraction

Required behavior

When raw OCR is noisy, do not stop at extraction.

• Keep the raw candidate layer for traceability.
• Prefer readability over raw OCR score when two candidates are close.
• Remove decorative fragments, isolated symbols, repeated garbage, and near-duplicate lines from the polished result.
• Keep uncertainty visible instead of pretending confidence.
• Never silently drop a major section when partial reconstruction is possible.
• Never present raw OCR dump as the final answer if a cleaner reconstruction can be produced.
• Preserve article structure when available: title, subtitle, author/source/time, heading levels, paragraphs, lists, captions, table-like rows, and appended notes.
• Treat information-bearing images as first-class content rather than an appendix afterthought.
• For image-heavy pages, support transcript-style and reading-order outputs in addition to clean article outputs.

WeChat / 公众号 handling

For mp.weixin.qq.com URLs:

• Try dedicated article extraction first when available.
• Fall back to static HTML parsing.
• Fall back again to browser rendering if needed.
• When the user cares about article readability, prefer reconstructing the final Word output in original reading order instead of appending all image OCR at the end.
• Use scripts/build_wechat_interleaved_docx.py when the task is specifically “keep original article order” for WeChat posts.
• If the page is blocked / validation-gated, report blocked: true clearly instead of pretending success.

Typical commands

Extract URL to markdown:

python3 {baseDir}/scripts/extract_visible_text.py \
  --url 'https://example.com/post' \
  --format markdown \
  --output result.md

Extract URL to JSON:

python3 {baseDir}/scripts/extract_visible_text.py \
  --url 'https://example.com/post' \
  --format json \
  --output result.json

Extract WeChat article with fallbacks:

python3 {baseDir}/scripts/extract_visible_text.py \
  --url 'https://mp.weixin.qq.com/s/xxxx' \
  --browser-fallback \
  --page-screenshot-ocr \
  --format markdown \
  --output wechat.md

Extract local screenshot or long image:

python3 {baseDir}/scripts/extract_visible_text.py \
  --image ./screenshot.png \
  --ocr-images \
  --format markdown \
  --output image-result.md

Run OCR post-processing:

python3 {baseDir}/scripts/postprocess_ocr_text.py \
  --input-json ./ocr-result.json \
  --title 'Clean Result' \
  --body-text 'Optional summary or body text' \
  --output-json ./clean.json \
  --output-markdown ./clean.md

Run the one-step deliverable pipeline:

python3 {baseDir}/scripts/extract_visible_text_deliverable.py \
  --url 'https://mp.weixin.qq.com/s/xxxx' \
  --browser-fallback \
  --page-screenshot-ocr \
  --ocr-images \
  --dedupe \
  --output-prefix ./deliverable/result

This should emit:

• result.raw.json
• result.clean.json
• result.clean.md
• result.docx

Run the already-authorized capture pipeline when the page can be opened in a browser or exported/saved first:

python3 {baseDir}/scripts/build_authorized_capture_docx.py \
  --url 'https://example.com/page' \
  --browser-capture \
  --ocr-images \
  --dedupe \
  --output-prefix ./deliverable/captured

Useful cases:

• browser can open the page but direct fetch is incomplete
• user provides a saved HTML page plus screenshots
• user wants one command that turns visible page content into a Word document
• user wants status visibility instead of silent long waits

Operational expectations for this pipeline:

• print stage logs so long OCR jobs do not look stuck
• fail loudly if expected outputs are not created
• detect obvious WeChat validation/interstitial text early
• optionally send the generated docx back to Feishu in one run
• when a source is blocked, stop pretending and switch to authorized-input workflows: saved HTML, screenshots, long images, copied text

Practical optimization rule:

• do not keep hammering a blocked source in the same mode
• if browser/direct fetch returns validation text, pivot immediately to the best authorized artifact path
• prioritize delivery quality: visible content captured by the user is better than repeated blocked fetch attempts

Key options

• --url webpage URL
• --text-file local plain text / markdown input
• --html-file local saved HTML page
• --image PATH add one local image or GIF; repeat as needed
• --image-dir DIR OCR all supported images / GIFs in a directory
• --format markdown|json output format
• --output PATH output file path
• --ocr-images OCR discovered or provided images
• --dedupe deduplicate repeated merged lines
• --browser-fallback use browser-rendered fallback for incomplete pages
• --page-screenshot-ocr OCR the browser full-page screenshot as a last resort
• --gif-mode none|placeholder conservative GIF handling mode

Quality standard

Default target: produce something a human can read comfortably and share without cleanup.

Release-quality target for article deliverables:

• preserve the article's original reading order whenever the source structure allows it
• avoid dumping all image OCR at the end when images belong in the middle of the article
• prefer a comfortable reading experience over a mechanically grouped OCR appendix
• keep English-heavy charts, dashboards, and mixed Chinese-English figures readable enough that key labels, axes, legends, and result summaries survive extraction

The skill should increasingly treat extraction as a full article understanding and recovery problem, not only a body scrape plus OCR problem:

• recover visible article structure from normal webpages, WeChat posts, blogs, tutorials, and mixed-media articles
• infer whether an image is mainly a price/product page, rules page, poster/event page, course outline, scenery/introduction card, or table-like detail page
• pull out high-value facts first when the user wants a clean readable result
• preserve near-complete text when the user wants transcript completeness
• avoid raw OCR dumps as the main deliverable unless the user explicitly wants audit output

When the user explicitly wants completeness, the skill must support a fuller extraction mode:

• treat each discovered image as a first-class source
• prefer segmented OCR for tall or dense images
• preserve near-complete per-image text blocks before compressing into summaries
• keep summary and full-text layers separate instead of replacing one with the other
• support reading-order transcript output so text and image-derived content can be followed from start to finish

For clean article outputs, prefer a structure like:

1. Title
2. Metadata (author/source/time) when meaningful
3. Main sections in order
4. Integrated image-derived supplements where needed
5. Uncertainty notes only when necessary

For transcript outputs, prefer a structure like:

1. Title
2. Intro/body chunks in order
3. Image text blocks in order or reading order
4. Tail matter / credits / appended notes

Mature-skill rule:

• default users toward the clean markdown / docx outputs unless they ask for transcript completeness
• keep raw JSON for audit, not as the main deliverable
• degrade honestly when the source is blocked or image quality is poor
• do not optimize only for one article family; keep checking travel-deal posts, rule/scoring posts, event posters, news/blog/tutorial pages, and course-outline long images

Read these references when needed:

• references/output-schema.md
• references/deliverable-workflow.md
• references/troubleshooting.md
• references/product-positioning.md
• references/generalization-plan.md
• references/universal-article-extractor-spec.md

Environment notes

• OCR depends on the local ocr-local skill or compatible Tesseract.js setup.
• Browser fallback depends on real browser availability plus playwright-core support.
• GIF frame extraction depends on ffmpeg.
• Some pages remain partially inaccessible due to login, anti-bot, or validation flows; mark those limits explicitly.