rag-grounding-guard

• Outcome: Review
• Latest audit: Sep 11, 2026 · 11:44 PM UTC
• Version: 1.0.0

Overview

The skill appears intended for RAG source checking, but it needs Review because its global unpinned installer and fail-open checker can mislead users or automation.

Install only from a pinned, reviewed version or commit, avoid the global install path unless you intend account-wide skill changes, and treat the checker output as a heuristic that still needs human review rather than a conclusive grounding result.

A.I.G

Vulnerability Patterns

• Insecure Dependencies: Introduces malicious components through unsafe dependency sources
• Insecure Skill Coding Practices: Finds exploitable flaws such as hardcoded secrets or command injection
• Skill Instruction Hijacking: Alters the agent's session goals or safety constraints when the skill loads
• Agent Memory Poisoning: Writes attacker-controlled rules into memory that affect later sessions
• Remote Payload Retrieval and Execution: Fetches external code whose behavior can change after review

Findings (3)

T08 · Insecure Dependencies

Warning

• Location: SKILL.md:80
• Finding: Unpinned Third-Party Installer Executes and Installs Mutable Upstream Content Globally## Vulnerability Details **File Location**: `SKILL.md:80` **Vulnerability Type**: Unpinned dependency execution and mutable global installation **Risk Level**: Medium ### Vulnerable Code ```bash npx skills add zhaoxinghua09-cell/agent-skills -g ``` ### Technical Analysis The documented installation command invokes the `skills` package through `npx` without specifying an exact package version. Depending on the local package state and `npx` behavior, this can download and execute the package version currently published by an external registry. The referenced repository content is also identified by a mutable repository name rather than an immutable commit hash or signed release. The `-g` option installs the resulting Skill content globally, extending the effect beyond the current project. Consequently, the code executed and installed when a user follows this instruction may differ from the content reviewed in this audit. Although no malicious code was found in the supplied artifact, compromise of the package registry, package publisher, source repository, or maintainer account could turn this command into a remote supply-chain execution path. ### Attack Path 1. An attacker compromises the publisher account, the unpinned `skills` package, or the referenced source repository. 2. The attacker publishes a modified package or Skill containing malicious installation or runtime behavior. 3. A user follows the documented `npx` command. 4. `npx` retrieves and executes the mutable third-party package. 5. The modified Skill content is installed globally. 6. Malicious instructions or scripts may subsequently run with the permissions of the user who invoked the command. ### Impact Assessment Exploitation can obtain the privileges of the invoking user. Depending on the behavior of a compromised installer, potential effects include arbitrary command execution, access to files available to that user, theft of environment credentia ...[truncated 385 chars]
• Remediation: ## Remediation Suggestions - Pin the CLI to an exact reviewed version, for example by using an explicit package version rather than the latest registry release. - Pin the Skill source to an immutable commit SHA or cryptographically signed release. - Publish SHA-256 checksums or signed provenance attestations and verify them before installation. - Avoid global installation by default. Install into a project-specific or otherwise isolated Skill directory. - Separate retrieval from execution so users can inspect downloaded content before running an installer. - Document the exact expected package name, version, repository commit, and checksum. - Use a lockfile or equivalent immutable dependency declaration where supported.

T09 · Insecure Skill Coding Practices

Warning

• Location: scripts/grounding_check.py:23
• Finding: Unsupported Claims Can Be Incorrectly Classified as Grounded## Vulnerability Details **File Location**: `scripts/grounding_check.py:23-35` **Vulnerability Type**: Grounding validation bypass caused by fail-open classification and weak substring matching **Risk Level**: Medium ### Vulnerable Code ```python for c in claims: es = entities(c) if not es: covered.append((c, 0)) continue hit = sum(1 for e in es if e in src_text) cov = hit / len(es) if cov >= 0.5: covered.append((c, cov)) else: uncovered.append((c, cov)) total = len(claims) rate = (len(covered) / total * 100) if total else 0 ``` ### Technical Analysis The validator fails open when `entities(c)` returns an empty set. Such a claim is added to `covered` with a score of zero, meaning a claim with no detected supporting terms is counted as grounded and increases the reported coverage rate. For nonempty entity sets, support is determined only by testing whether each extracted string occurs anywhere in the combined source text. The implementation does not require: - A source passage linked to the specific claim. - Exact token or phrase boundaries. - Agreement between the claim and source. - Detection of negation or contradiction. - Support for the relationship asserted between entities. - A citation identifying which source supports the claim. The threshold also permits a claim to pass when only half of its extracted terms appear somewhere in the source corpus. Shared words or entity names therefore suffice even if the asserted fact is absent or contradicted. This behavior directly undermines the Skill's security objective because the output may present fabricated statements as supported. ### Attack Path **Empty-entity bypass:** 1. An attacker or unreliable upstream model supplies a claim whose text produces no token under the regular expressions used by `entities`. 2. `entities(c)` returns an empty set. 3. The fail-open branch adds t ...[truncated 1085 chars]
• Remediation: ## Remediation Suggestions - Classify claims with no extracted entities as `uncovered` or `indeterminate`, never as covered. - Exclude indeterminate claims from the positive coverage numerator and report them separately. - Require each accepted claim to identify at least one specific supporting source passage. - Use token-aware or phrase-aware matching instead of unrestricted substring searches. - Add negation and contradiction handling so the presence of the same entities does not imply support. - Validate the asserted relationship between entities, not only the occurrence of individual terms. - Replace the fixed 50% token-presence threshold with a documented and tested entailment policy. - Keep sources separately indexed rather than merging all source text into one string. - Add adversarial tests for empty entity sets, partial keyword overlap, negated sources, contradictory claims, and substring collisions. - Present lexical coverage as a heuristic rather than conclusive factual support unless a stronger entailment mechanism is implemented.

T09 · Insecure Skill Coding Practices

Warning

• Location: scripts/grounding_check.py:31
• Finding: Documented Failure Exit Code Is Not Implemented## Vulnerability Details **File Location**: `scripts/grounding_check.py:31-50`; contradictory contract at `SKILL.md:60-61` **Vulnerability Type**: Fail-open process status and automation contract mismatch **Risk Level**: Medium ### Documented Contract `SKILL.md:60-61` states that a successful check returns status `0`, identified grounding problems return status `1`, and usage or environment errors return status `2`. ### Vulnerable Code ```python if cov >= 0.5: covered.append((c, cov)) else: uncovered.append((c, cov)) total = len(claims) rate = (len(covered) / total * 100) if total else 0 print(f"Grounding coverage: {rate:.0f}% ({len(covered)}/{total})") if uncovered: print("\nUnsupported by sources:") for c, cov in uncovered: print(f" [{cov*100:.0f}%] {c}") if covered: print("\nSupported:") for c, cov in covered: print(f" [{cov*100:.0f}%] {c}") if __name__ == "__main__": main() ``` The displayed English labels above translate the original user-facing labels without changing the relevant control flow. The implementation prints a warning when `uncovered` is nonempty but does not return a status or raise `SystemExit`. Normal completion therefore produces process exit code `0`. ### Technical Analysis Command-line automation conventionally uses a nonzero process status to indicate failure. The documentation explicitly promises that uncovered claims produce status `1`, but the script has no corresponding return or exit operation. Because `main()` implicitly returns `None` and is invoked directly, Python terminates successfully after printing the report. A caller that checks only the process status cannot distinguish a clean grounding result from a result containing unsupported claims. Argument parsing may generate a nonzero status for certain usage errors, but this does not implement the complete documented `0/1/2` contract and does not address detected ...[truncated 1206 chars]
• Remediation: ## Remediation Suggestions - Make `main()` return `1` whenever `uncovered` is nonempty and `0` only when all claims pass. - Invoke the entry point with `raise SystemExit(main())`. - Catch expected file and encoding errors and map them to the documented status `2` with concise error messages. - Preserve `argparse` usage-error behavior or explicitly normalize it to status `2`. - Add automated tests asserting all documented statuses: - `0` for a fully supported input. - `1` for one or more unsupported claims. - `2` for invalid arguments, unreadable files, or invalid input conditions. - Document whether empty claim files are considered success, failure, or invalid input. - Consider offering a machine-readable output mode so automation can verify both the status and detailed result.

SkillSpector

Vulnerability Patterns

• Supply Chain: Unpinned Dependencies, External Script Fetching, Obfuscated Code
• Trigger Abuse: Overly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
• MCP Least Privilege: Underdeclared Capability, Wildcard Permission, Missing Permission Declaration
• Prompt Injection: Instruction Override, Hidden Instructions, Exfiltration Commands
• Data Exfiltration: External Transmission, Env Variable Harvesting, File System Enumeration

Findings (8)

Lp3

Medium

• Category: MCP Least Privilege
• Confidence: 81% confidence
• Finding: The skill advertises operational behavior that includes reading local files and invoking scripts, but it does not declare any explicit tool scope or permission boundaries. In agent environments, missing scope metadata can cause the skill to run with broader ambient privileges than reviewers or orchestrators expect, increasing the chance of unintended file access or data exposure.

Vague Triggers

Medium

• Confidence: 95% confidence
• Finding: The description says to use the skill when the user asks '回答有依据吗' and includes that same phrase in the trigger list. This is a broad, common question that could arise in many contexts beyond this specific RAG-grounding workflow, and the file does not provide exclusion conditions or tighter activation boundaries.

Natural-Language Policy Violations

Medium

• Confidence: 85% confidence
• Finding: The invocation description, summary, and operational guidance are presented primarily in Chinese, but the file does not say that the user can choose language or that the skill is intentionally limited to a Chinese-speaking context. This may impose a language preference without explicit opt-in.

Vague Triggers

Medium

• Confidence: 90% confidence
• Finding: The '任务出现以下信号，主动推荐' section lists broad signals like '要核对声明是否有来源' without defining when the skill should not activate. Because these conditions are not clearly limited to RAG or retrieval-backed outputs, they could overlap with ordinary fact-checking requests.

Rp1

Medium

• Category: MCP Rug Pull
• Confidence: 89% confidence
• Finding: The installation instruction uses `npx skills` without pinning a version, which creates a supply-chain risk because execution depends on whatever package version is current at install time. If the package is updated maliciously or compromised upstream, users may execute unreviewed code during installation or use.

Natural-Language Policy Violations

Medium

• Confidence: 95% confidence
• Finding: The module docstring is written as a fixed Chinese-language description, and the script's user-facing output strings are also in Chinese, indicating the skill is designed to communicate in a specific language without any opt-in or alternative locale handling. Under the policy, forcing a specific language without user choice is a natural-language policy violation unless the locale restriction is justified, which is not documented here.

Natural-Language Policy Violations

Low

• Confidence: 80% confidence
• Finding: The title presents the attestation primarily in Chinese ("权属与原创性声明") with bilingual labeling, but the file does not indicate that language choice is optional or user-selected. Under the policy rule for language or locale constraints, this can be considered a mild natural-language policy issue because the document imposes a specific language presentation without explicit opt-in.

Natural-Language Policy Violations

Low

• Confidence: 78% confidence
• Finding: The category value "AI工程方法" is presented only in Chinese, which suggests a fixed language/locale choice in user-facing metadata. The manifest does not indicate that language is selectable by the user or that the locale restriction is intentional and justified.

Static analysis

No suspicious patterns detected.