Security audit

Sitemap Generator

Security checks for vulnerabilities and agentic risk

Overview

This is a coherent sitemap generator, but its crawler silently disables HTTPS verification and can be redirected outside the promised same-domain boundary.

Review before installing. Use it only when you intend to let the agent crawl a specified site or scan a chosen local directory, and avoid using it on sensitive internal sites or untrusted networks until TLS verification and redirect destination validation are fixed. Check output paths because --output and --robots write files in the workspace.

A.I.G

Vulnerability Patterns

Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally

Findings (2)

T09 · Insecure Skill Coding Practices

Error

Location: scripts/sitemap_gen.py:72
Finding: TLS Certificate and Hostname Verification Disabled<![CDATA[ ## Vulnerability Details **File Location**: `scripts/sitemap_gen.py`, lines 72–74 **Vulnerability Type**: Improper TLS certificate validation **Risk Level**: High ### Vulnerable Code ```python ctx = ssl.create_default_context() ctx.check_hostname = False ctx.verify_mode = ssl.CERT_NONE ``` ### Technical Analysis The crawler explicitly disables both TLS certificate-chain validation and hostname verification for every HTTPS request. Consequently, it accepts expired, self-signed, untrusted, and hostname-mismatched certificates. HTTPS encryption without certificate validation does not authenticate the remote server. An attacker capable of intercepting network traffic can impersonate the requested website and return attacker-controlled HTML. Because the crawler parses links from successful HTML responses, the attacker can manipulate discovered pages and generated sitemap content. ### Attack Path 1. A user invokes the Skill to crawl an HTTPS website. 2. An attacker gains a network interception position, such as through a malicious access point, compromised proxy, DNS manipulation, or local network control. 3. The attacker presents an arbitrary TLS certificate for the requested hostname. 4. The crawler accepts the certificate because certificate and hostname checks are disabled. 5. The attacker supplies modified HTML containing selected same-domain links and query strings. 6. The crawler parses those links, makes additional requests, and incorporates successful URLs into the generated sitemap. ### Impact Assessment The attacker does not directly obtain operating-system privileges through this flaw. However, an on-path attacker can: - Impersonate any HTTPS website crawled by the tool. - Read requested paths and query strings. - Modify crawler responses and discovered URL sets. - Inject misleading or attacker-selected URLs into sitemap, text, or JSON output. - Influence subsequent network requests made by the crawler. - Undermine the confidentiality ...[truncated 117 chars]
Remediation: <![CDATA[ ## Remediation Suggestions Retain Python's secure default TLS behavior: ```python ctx = ssl.create_default_context() ``` Do not assign `ssl.CERT_NONE`, and do not disable `check_hostname`. The request can then continue to use the verified context: ```python resp = urllib.request.urlopen(req, timeout=timeout, context=ctx) ``` Additional hardening measures include: 1. Fail closed when certificate validation fails. 2. Return or log a clear TLS validation error without silently retrying insecurely. 3. Use the operating system's trusted certificate store. 4. If private development certificates must be supported, allow the user to provide an explicit CA bundle rather than disabling verification globally. 5. If an insecure testing mode is considered unavoidable, require an explicit command-line option, display a prominent warning, and ensure it is disabled by default. 6. Add automated tests confirming rejection of self-signed, expired, and hostname-mismatched certificates. ]]>

T09 · Insecure Skill Coding Practices

Warning

Location: scripts/sitemap_gen.py:79
Finding: Automatic Redirects Bypass the Same-Domain Crawl Restriction<![CDATA[ ## Vulnerability Details **File Location**: `scripts/sitemap_gen.py`, lines 79–87 and 127–136 **Vulnerability Type**: Unvalidated redirect resulting in an SSRF-like domain-boundary bypass **Risk Level**: Medium ### Vulnerable Code The request function follows redirects through `urllib.request.urlopen` and does not inspect the final response URL: ```python headers = {"User-Agent": "SitemapGenerator/1.0 (+https://clawhub.com/skills/sitemap-generator)"} req = urllib.request.Request(url, headers=headers) try: resp = urllib.request.urlopen(req, timeout=timeout, context=ctx) content_type = resp.headers.get("Content-Type", "") if "text/html" not in content_type: return resp.getcode(), "", content_type content = resp.read().decode("utf-8", errors="replace") return resp.getcode(), content, content_type ``` The same-domain validation is performed only before that request: ```python if not is_same_domain(url, base_domain): continue if should_skip(url): continue if verbose: print(f" Crawling: {url}", file=sys.stderr) status, content, ctype = fetch_page(url, timeout=timeout) if status and 200 <= status < 400: ``` ### Technical Analysis `urllib.request.urlopen` follows HTTP redirects automatically. The crawler validates the queued URL against `base_domain` before calling `fetch_page`, but it neither validates each redirect target nor checks `resp.geturl()` after redirects have been followed. A URL on the approved domain can therefore redirect the request to: - An unrelated external host. - A loopback service such as `127.0.0.1`. - A private-network address. - A link-local service or cloud instance metadata endpoint. The request is made from the machine running the Skill, so its network reachability may exceed that of an external attacker. This behavior also contradicts the documented “same-domain only” crawl boundary. The final body is processed only when its content type contains `text/html`, which limits d ...[truncated 1663 chars]
Remediation: <![CDATA[ ## Remediation Suggestions Enforce destination policy at every redirect hop rather than only on the initial queued URL. Recommended controls: 1. Disable automatic redirects with a custom `HTTPRedirectHandler`. 2. Parse and normalize every `Location` value with `urljoin`. 3. Reject a redirect unless its scheme is explicitly permitted and its normalized hostname and effective port remain within the approved origin policy. 4. Apply a small redirect limit to prevent redirect loops. 5. After opening a response, inspect `resp.geturl()` and reject it if it differs from the permitted origin. 6. Resolve destination hostnames and reject loopback, private, link-local, multicast, unspecified, and reserved IP address ranges. 7. Revalidate every redirect after DNS resolution to reduce DNS rebinding risk. 8. Consider restricting crawl mode to public HTTP and HTTPS destinations unless private-network crawling is explicitly requested. 9. Add tests for redirects to another hostname, `localhost`, IPv4 and IPv6 loopback addresses, RFC 1918 ranges, link-local addresses, and multi-hop redirect chains. A final-response check should be used as defense in depth: ```python resp = urllib.request.urlopen(req, timeout=timeout, context=ctx) final_url = normalize_url(resp.geturl()) if not is_same_domain(final_url, base_domain): resp.close() return None, "", "" ``` This check alone occurs after the unauthorized request and therefore does not fully prevent SSRF. Redirect handling must reject disallowed targets before contacting them. ]]>

SkillSpector

Vulnerability Patterns

Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands

Findings (5)

Missing User Warnings

High

Confidence: 99% confidence
Finding: HTTPS requests are made with authentication of the remote endpoint disabled and no warning to the user. In a security-sensitive crawling context, this means network attackers can tamper with responses, causing the tool to trust and process spoofed HTML as if it came from the intended domain.

Lp3

Medium

Category: MCP Least Privilege
Confidence: 97% confidence
Finding: The skill instructs the agent to perform network access, read local files, and write output files, but it does not declare any explicit tool scope or permissions boundaries. In an agent environment, this increases the chance of unintended website crawling or filesystem writes occurring without clear user consent or policy enforcement.

Vague Triggers

Medium

Confidence: 94% confidence
Finding: The trigger phrases are broad enough to match common requests like "robots.txt" or "sitemap," which may invoke this skill in contexts where the user did not intend live crawling or file generation. Overbroad activation can cause the agent to select a higher-risk skill than necessary, leading to unexpected network requests or filesystem modifications.

Missing User Warnings

Medium

Confidence: 94% confidence
Finding: The skill documentation does not clearly warn that it may crawl live websites and write files like sitemap.xml and robots.txt. Without explicit disclosure, users or orchestrators may not realize that using the skill can initiate external network activity and modify the local workspace, which raises consent and safety concerns.

Context-Inappropriate Capability

Medium

Confidence: 99% confidence
Finding: The crawler explicitly disables both TLS certificate validation and hostname verification before fetching HTTPS pages. This allows a man-in-the-middle attacker to impersonate any target site, alter crawled content, and inject arbitrary links into the generated sitemap without detection.

Static analysis

Detected: suspicious.insecure_tls_verification

HTTPS certificate verification is disabled.

Warn

Code: suspicious.insecure_tls_verification
Location: scripts/sitemap_gen.py:76