T09 · Insecure Skill Coding Practices
- Location
scripts/scrape.py:39- Finding
Unrestricted URL Fetching Enables Server-Side Request Forgery
- Content
View full analysis
- Remediation
View remediation
Security audit
Security checks for vulnerabilities and agentic risk
This web-scraping skill has a coherent purpose, but its anti-bot bypass guidance, weak robots.txt handling, unrestricted URL fetching, and persistent local learning data need review before use.
Review this skill before installing in any environment with access to private networks, cloud metadata, authenticated web sessions, or sensitive local paths. Use it only for authorized public scraping, avoid the anti-detection guidance unless you have explicit permission, and treat learned_patterns.json as persistent plaintext usage history.
scripts/scrape.py:39Unrestricted URL Fetching Enables Server-Side Request Forgery
scripts/scrape.py:39robots.txt Enforcement Fails Open on Retrieval or Parsing Errors
The skill presents itself as automatically respecting robots.txt and providing safe throttled scraping, yet the described behavior weakens that guarantee and omits key controls like strict rate limiting and validation. That gap can lead users to unintentionally run collection in ways that violate site policies or create unnecessary legal and operational risk.
The skill presents itself as automatically respecting robots.txt and providing safe throttled scraping, yet the described behavior weakens that guarantee and omits key controls like strict rate limiting and validation. That gap can lead users to unintentionally run collection in ways that violate site policies or create unnecessary legal and operational risk.
The inclusion of stealth patches, residential proxy rotation, and behavior simulation is specifically aligned with defeating anti-bot and anti-abuse controls rather than ordinary public-data collection. In this context, those features materially increase the skill's utility for unauthorized scraping, detection evasion, and abuse at scale.
The skill declares capabilities that imply network access and writing files, but it does not constrain tool scope with explicit permissions or allowed-tools. In an agent environment, that increases the blast radius: the skill could be invoked with broader-than-necessary access and save collected or learned data to disk without clear governance.
The skill advertises automatic respect for robots.txt, but its usage documentation explicitly supports skipping robots checks. That discrepancy broadens the behavior beyond the declared compliant scope and makes misuse easier by exposing a ready-made bypass path.
The manifest description is entirely in Chinese and does not indicate that other languages are supported or that the user can opt into a preferred language. This can violate language/locale policy when a skill unconditionally presents itself in a specific language.
The main skill instructions and examples are presented exclusively in Chinese, with no indication that the language is optional or region-specific. Under the policy, forcing a specific language without user opt-in is a natural-language policy concern.
The safety section claims strict compliance with robots.txt and ToS, but the same document provides a flag to skip robots checks and guidance for evading anti-bot defenses. This contradiction normalizes policy bypass and can facilitate unauthorized scraping while misleading reviewers into believing the skill is constrained.
The file introduces a generic cross-skill 'self-evolution' module inside a web-fetch skill, with support for arbitrary skill directories and storage under ~/.workbuddy/skills. That exceeds the declared scraping scope and creates an unnecessary capability for collecting and persisting metadata across skills, increasing privacy and trust-boundary risk even if it is not overtly malicious.
The code persistently records preferences, recent operations, error notes, and capability history across runs in learned_patterns.json. Because note/error fields may contain sensitive user-provided or operational content and resolve_skill_dir permits targeting arbitrary skill directories, this creates a privacy and data-minimization issue not required for the advertised web-scraping function.
save_learning writes preference and usage data to disk without any explicit notice, consent flow, or sensitivity guidance. Silent persistence is risky because users may unknowingly store behavioral history or sensitive notes in plaintext JSON on the local filesystem.
On robots.txt read or parse failure, the code returns True and proceeds with scraping, while the comment incorrectly describes this as a conservative default. This fail-open behavior can cause unauthorized scraping when robots policies cannot be retrieved due to network errors, blocking, or parser issues.
The code silently allows scraping when robots.txt checks fail, with no user-facing warning that policy validation did not occur. In the context of an automation-oriented scraping skill, this increases the chance of noncompliant collection at scale because operators may wrongly assume robots rules were enforced.
The script exposes a --no-robots flag that explicitly disables robots.txt enforcement, contradicting the skill description that it 'automatically respects robots.txt'. In a web-scraping skill, this materially lowers safety controls and enables users or downstream agents to scrape sites that have declared disallow rules.
The natural-language interface shown in the module docstring, help descriptions, and printed messages is Chinese-only. Under the policy, forcing a specific language without offering a user choice can be a locale/language policy violation unless clearly justified as region-specific.
The module docstring and command help/messages are presented in Chinese only, which imposes a specific language on users without any stated opt-in or alternative. This is a natural-language policy concern because the skill does not offer locale choice or document that it is intentionally limited to a Chinese-speaking audience.
No suspicious patterns detected.