LLMs.txt Generator
PassAudited by ClawScan on May 1, 2026.
Overview
This skill appears to do what it advertises: crawl a user-specified website and draft an llms.txt file, with normal web-crawling and setup considerations.
Before installing, be aware that the skill runs a website crawler, may place crawled page text into the model context, and may require manual Python dependency setup. Review the URLs being crawled and check the final llms.txt before publishing it.
Findings (3)
Artifact-based informational review of SKILL.md, metadata, install specs, static scan signals, and capability signals. ClawScan does not execute the skill or run runtime probes.
The agent will make web requests to the provided site and store extracted page data locally while preparing the llms.txt draft.
The skill instructs the agent to run a local crawler on a user-provided URL and write crawl results to a temporary file. This is central to the skill’s purpose, but it is still a user-visible network and local-command capability.
~/.virtualenvs/llms-txt-generator/bin/python3 \
~/.openclaw/workspace/llms-txt-generator/scripts/crawl.py \
{url} > /tmp/llms_business_info.jsonUse it only with websites you intend to crawl, avoid internal/private URLs unless that is deliberate, and review the generated draft before publishing.
If a crawled page contains misleading or adversarial text, it could influence the draft llms.txt content.
In deep mode, untrusted webpage text is placed into the model context for extraction. That is expected for this generator, but webpage content should be treated as data rather than instructions.
This returns `pages_raw` — the full raw text of every crawled page. Use it to extract structure with the LLM.
Verify the final llms.txt manually, especially team, pricing, testimonial, API, and contact details extracted from crawled pages.
A user or environment may need to install dependencies manually, which can introduce version or provenance uncertainty.
The crawler depends on Python packages that are not declared in the package metadata or an install spec. This is a setup/provenance note, not evidence of malicious behavior.
Missing dependencies. Run: pip install httpx beautifulsoup4 lxml
Prefer a pinned requirements file or trusted environment setup before running the crawler.
