Back to skill

Security audit

cite-holmes

Security checks for vulnerabilities and agentic risk

Overview

The skill is mostly a coherent citation-checking research helper, but it needs Review because its verifier can contact arbitrary URLs and can overstate citation verification in some modes.

Review before installing if you handle sensitive research, medical topics, or untrusted reference lists. Run the verifier in a restricted network environment, avoid processing attacker-supplied URLs, treat offline results as structural checks only, and manually review medical citations until private-address blocking, fail-closed PMID checks, and stricter tier derivation are fixed.

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (3)

T09 · Insecure Skill Coding Practices

Warning
Location
scripts/verify_refs.py:164
Finding
Arbitrary Reference URLs Enable Server-Side Request Forgery<![CDATA[ ## Vulnerability Details **File Location**: `scripts/verify_refs.py:164-172`, with attacker-controlled input reaching the request at `scripts/verify_refs.py:219-239` **Vulnerability Type**: Server-Side Request Forgery through unrestricted outbound HTTP requests **Risk Level**: Medium ### Vulnerable Code ```python def check_url(url: str, timeout: float) -> tuple: """返回 (reachable, status, note)。reachable 以 2xx/3xx/429(反爬) 计。""" req = urllib.request.Request(url, method="HEAD", headers={ "User-Agent": "Mozilla/5.0 (compatible; cite-holmes/1.0; +verified-deep-research)", "Accept": "*/*", }) opener = urllib.request.build_opener(urllib.request.HTTPRedirectHandler()) for method in ("HEAD", "GET"): try: req.method = method with opener.open(req, timeout=timeout) as resp: return True, resp.status, f"{method} {resp.status}" ``` The URL originates from a user-controlled reference record: ```python url = ref_to_url(ref) if not url: out.update(verdict="invalid", note="缺少 url 且无可解析的 doi/pmid" + ("(pmid 须为 6-9 位数字)" if pmid else "")) return out if not re.match(r"^https?://", url): out.update(verdict="invalid", note=f"url 非 http(s) 格式: {url[:60]}") return out if not out["tier"]: out["tier"] = classify_tier(url, medical) out["url"] = url if medical and out["tier"] in ("community", "social", "blog"): out["note"] = "医学模式:社区/社交/博客层来源不得支撑医学结论(仅作线索)" if offline: out["note"] = (out["note"] + ";" if out["note"] else "") + "offline 模式未做可达性检查" else: reachable, status, note = check_url(url, timeout) ``` ### Technical Analysis The verifier accepts arbitrary HTTP or HTTPS URLs from reference JSON and sends outbound `HEAD` requests, followed by `GET` requests when necessary. It does not resolve and reject loopback, private, link-local, reserved, multicast, or unspecified IP addresses. It also enables automatic redirects witho ...[truncated 1877 chars]
Remediation
<![CDATA[ ## Remediation Suggestions 1. Resolve the destination hostname before connecting and reject every address in loopback, private, link-local, multicast, reserved, and unspecified ranges for both IPv4 and IPv6. 2. Permit only HTTPS by default; if HTTP is required, make it an explicit opt-in. 3. Restrict destination ports to 80 and 443. 4. Disable automatic redirects or implement a custom redirect handler that revalidates the scheme, hostname, resolved addresses, and port at every hop. 5. Limit the number of redirects and reject URL credentials and unusual hostname representations. 6. Apply DNS rebinding protections by validating the address immediately before connection and, where possible, binding the request to the validated address. 7. Consider a public-domain allowlist or an isolated outbound proxy for citation checks. 8. Add regression tests covering loopback, RFC 1918 ranges, link-local addresses, IPv6 local ranges, encoded IP addresses, and redirects from public hosts to private destinations. ]]>

T09 · Insecure Skill Coding Practices

Warning
Location
scripts/verify_refs.py:211
Finding
Caller-Controlled Source Tiers and Offline Mode Can Produce False Verified Verdicts<![CDATA[ ## Vulnerability Details **File Location**: `scripts/verify_refs.py:211-263` **Vulnerability Type**: Verification bypass and trust-boundary violation **Risk Level**: Medium ### Vulnerable Code ```python def verify_one(ref: dict, idx: int, offline: bool, timeout: float, medical: bool = False) -> dict: out = {"index": idx, "title": ref.get("title") or "(无标题)", "url": ref.get("url") or "", "doi": str(ref.get("doi") or "").strip(), "pmid": str(ref.get("pmid") or "").strip(), "source": ref.get("source") or "", "year": ref.get("year"), "tier": ref.get("tier") or "", "semantic": ref.get("semantic", ""), "verdict": "unverified", "http_status": None, "note": "", "needs_human_check": False} pmid = out["pmid"] url = ref_to_url(ref) if not url: out.update(verdict="invalid", note="缺少 url 且无可解析的 doi/pmid" + ("(pmid 须为 6-9 位数字)" if pmid else "")) return out if not re.match(r"^https?://", url): out.update(verdict="invalid", note=f"url 非 http(s) 格式: {url[:60]}") return out if not out["tier"]: out["tier"] = classify_tier(url, medical) out["url"] = url if medical and out["tier"] in ("community", "social", "blog"): out["note"] = "医学模式:社区/社交/博客层来源不得支撑医学结论(仅作线索)" if offline: out["note"] = (out["note"] + ";" if out["note"] else "") + "offline 模式未做可达性检查" else: reachable, status, note = check_url(url, timeout) out["http_status"] = status out["note"] = (out["note"] + ";" if out["note"] else "") + note if not reachable: out.update(verdict="unreachable", needs_human_check=True, note=(out["note"] + ";" if out["note"] else "") + "可能反爬/临时故障,不等于不存在") return out m = PUBMED_URL_RE.match(url) if m: pmid_ok, pmid_note = pubmed_pmid_exists(m.group(1), timeout) if not pmid_ok: ...[truncated 2792 chars]
Remediation
<![CDATA[ ## Remediation Suggestions 1. Treat input records as untrusted and ignore the supplied `tier` during normal verification. 2. Derive the tier exclusively from the validated final destination URL. 3. If manual tier overrides are operationally necessary, require a separate trusted configuration file or explicit administrative flag and label the result as manually asserted. 4. In offline mode, always retain the verdict `unverified` after structural validation; never produce `verified`. 5. Separate structural validity, reachability, source authority, and semantic support into distinct fields rather than collapsing them into one verdict. 6. Prevent unverified offline records from entering verified-only BibTeX exports. 7. Add tests proving that: - A supplied trusted tier cannot override automatic classification. - Every structurally valid offline item remains `unverified`. - Offline records do not increase the verified count or enter verified-only exports. 8. Update the CLI output and documentation so offline mode is clearly identified as structural validation only. ]]>

T09 · Insecure Skill Coding Practices

Warning
Location
scripts/verify_refs.py:139
Finding
PubMed Existence Verification Fails Open on NCBI Errors<![CDATA[ ## Vulnerability Details **File Location**: `scripts/verify_refs.py:139-159`, with the result trusted at `scripts/verify_refs.py:241-249` **Vulnerability Type**: Fail-open external validation **Risk Level**: Medium ### Vulnerable Code ```python def pubmed_pmid_exists(pmid: str, timeout: float) -> tuple: """经 NCBI E-utilities 核实 PMID 真实存在。 PubMed 网页对不存在的 PMID 也返回 2xx/203(且带反爬壳页),HTTP 状态码 无法区分真假——AI 编造的 PMID 必须靠 API 核实才能抓住。 返回 (exists, note)。 """ u = ("https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esummary.fcgi" f"?db=pubmed&id={pmid}&retmode=json") try: req = urllib.request.Request( u, headers={"User-Agent": "cite-holmes/1.1; +verified-deep-research"}) with urllib.request.urlopen(req, timeout=timeout) as resp: j = json.loads(resp.read().decode("utf-8", "ignore")) res = j.get("result") or {} item = res.get(str(pmid)) or {} if item.get("error") or str(pmid) not in (res.get("uids") or []): return False, "PMID 在 PubMed 不存在(E-utilities 核实)→ 疑似编造引用" return True, "PMID 经 E-utilities 核实存在" except Exception as e: return True, f"E-utilities 校验失败({type(e).__name__}),按可达处理" ``` The fail-open result is subsequently accepted as successful verification: ```python m = PUBMED_URL_RE.match(url) if m: pmid_ok, pmid_note = pubmed_pmid_exists(m.group(1), timeout) if not pmid_ok: out.update(verdict="invalid", needs_human_check=True, note=(out["note"] + ";" if out["note"] else "") + pmid_note) return out out["note"] = (out["note"] + ";" if out["note"] else "") + pmid_note ``` ### Technical Analysis The dedicated NCBI E-utilities check is intended to distinguish valid PMIDs from fabricated identifiers because PubMed pages can return successful HTTP responses for nonexistent records. However, every exception is converted into `(True, ...)`, which has the same Boolean meaning as confirmed exist ...[truncated 1618 chars]
Remediation
<![CDATA[ ## Remediation Suggestions 1. Never interpret an exception as proof that a PMID exists. 2. Return a distinct indeterminate state, such as `None`, or a structured result containing `exists`, `checked`, and `error`. 3. Map unavailable or inconclusive E-utilities responses to `unverified` or `unreachable`, with `needs_human_check=True`. 4. Assign `verified` only after a valid E-utilities response explicitly lists the requested PMID in `result.uids` and contains no record-level error. 5. Implement bounded retries with exponential backoff for transient failures and respect NCBI rate limits. 6. Validate HTTP status, content type, response size, JSON structure, and identifier consistency before accepting the response. 7. Prevent indeterminate PMID records from entering verified-only exports. 8. Add tests for timeout, DNS failure, malformed JSON, empty responses, HTTP rate limiting, and unexpected API schemas, confirming that each case fails closed. ]]>
Vulnerability Patterns
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
Findings (17)

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding
The declared description is for a sophisticated verified deep-research and citation-validation skill. The supplied code chunk does none of that. It imports matplotlib, defines drawing helpers, creates several composed vector-style illustrations, and saves them as PNG files in a temporary directory. There is no network access, no document or bibliography parsing, no source lookup, no verification logic, no language handling, and no research/report pipeline. This is a clear material mismatch in primary purpose and capabilities.

Ae1

High
Category
analysis-evasion
Content
python scripts/verify_refs.py --refs research_refs.json --out verify_report.md
Confidence
100% confidence
Finding
Referenced artifact was not completely inspected

Ae1

High
Category
analysis-evasion
Content
python scripts/verify_refs.py --refs research_refs.json --out verify_report.md
Confidence
100% confidence
Finding
Referenced artifact was not completely inspected

Ae1

High
Category
analysis-evasion
Content
python scripts/verify_refs.py --refs research_refs.json --out verify_report.md
Confidence
100% confidence
Finding
Referenced artifact was not completely inspected

Ae1

High
Category
analysis-evasion
Content
python scripts/verify_refs.py --refs research_refs.json --out verify_report.md
Confidence
100% confidence
Finding
Referenced artifact was not completely inspected

Vague Triggers

Medium
Confidence
95% confidence
Finding
The README says the skill 'triggers automatically' for a wide range of common research-style prompts, which creates an overly broad activation surface. In agent environments where skills may initiate web access, source retrieval, or extensive processing, this can cause unintended invocation on ambiguous user requests and lead to unnecessary data exposure, network activity, or surprise behavior.

Lp3

Medium
Category
MCP Least Privilege
Confidence
89% confidence
Finding
The skill explicitly instructs use of network access and writing output files (for verification reports and exports) but declares no tool scope or allowed-tools boundary. That omission weakens least-privilege controls and can lead to overbroad execution in hosts that rely on manifest-declared permissions, especially since the skill performs external lookups and writes artifacts to disk.

Vague Triggers

Medium
Confidence
92% confidence
Finding
The trigger text is very broad and overlaps with many ordinary requests such as fact-checking, comparing topics, investigating claims, or literature review. Over-broad invocation can cause the skill to activate unexpectedly, leading to unnecessary external searches, data transmission, and file generation in contexts where the user did not intend to invoke this workflow.

Missing User Warnings

Medium
Confidence
91% confidence
Finding
The medical verification flow states that PMIDs and PubMed references are existence-checked via NCBI E-utilities and other external sources, but the skill text does not clearly warn users that citation details may be transmitted to third-party services. In medical or academic settings, even bibliographic queries can reveal sensitive research interests, manuscript contents, or case-related topics, making the lack of disclosure a privacy and compliance concern.

Natural-Language Policy Violations

Medium
Confidence
85% confidence
Finding
The skill mandates bilingual medical searching using both English and Chinese sources ('英文 PubMed,中文知网/万方') without indicating that this should depend on the user's language, jurisdiction, or locale. In a medical-research workflow, forcing locale-specific sources can bias evidence selection, create compliance/privacy issues across regions, and cause the agent to present China-specific guidance as generally applicable.

Natural-Language Policy Violations

Medium
Confidence
95% confidence
Finding
The title and the entire document are written in Chinese, and the file does not indicate that this language choice is optional or limited to a region-specific use case. Under the policy, forcing a specific language without user opt-in is a natural-language locale violation.

Natural-Language Policy Violations

Medium
Confidence
92% confidence
Finding
This code file contains natural-language instructions, help text, and reporting strings that assume Chinese as the required interaction language. The policy only allows fixed language constraints when the skill offers user opt-in or clearly documents a justified locale restriction, which is not present here.

Natural-Language Policy Violations

Medium
Confidence
92% confidence
Finding
This Python test file contains natural-language content and assertions entirely in Chinese, including expected rendered output and notes, which implies the skill behavior is tied to a specific language/locale. Under the policy, forcing a specific language without user opt-in or clear documented regional justification is a violation.

Natural-Language Policy Violations

Low
Confidence
88% confidence
Finding
The README states the SEARCH phase uses "Chinese + English," which imposes a language/locale behavior by default. There is no indication that users can opt in, opt out, or choose different language settings, which conflicts with the language-choice policy for natural-language behavior.

Natural-Language Policy Violations

Low
Confidence
88% confidence
Finding
This JSON contains natural-language fields such as `claim` and `_demo_note` in Chinese while other user-facing fields like `title` and `source` are in English. Because the file provides no indication that the skill is intentionally Chinese-only or offers language selection, it can violate a language/locale policy requiring explicit opt-in or documented locale constraints.

Natural-Language Policy Violations

Low
Confidence
87% confidence
Finding
The markdown explicitly states that the report language must follow the user's question language. This imposes a locale/language behavior by policy rather than offering a choice, which matches the rule's language-policy concern for natural-language instructions.

Missing User Warnings

Low
Confidence
91% confidence
Finding
This code performs a file write to a repository asset path by saving a generated GIF, but the write operation has no confirmation prompt and no nearby user-facing warning before it occurs. For code files, file writes can be flagged when there is no disclosure in code comments, prompts, or visible messaging about the action.

Static analysis

No suspicious patterns detected.