ClawHub
Back to skill

Security audit

Hallucination Guard

Security checks for vulnerabilities and agentic risk

Overview

This skill is a legitimate verification aid, but it gives agents broad automatic command-running and file-modification behavior that needs careful review before installation.

Install only if you are comfortable with an agent running shell commands, URL checks, package registry lookups, and possible edits as part of verification. Use explicit target paths, prefer read-only or dry-run operation, disable arbitrary custom command checks from untrusted repositories, and approve any fixes before files are changed.

A.I.G

Vulnerability Patterns
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
  • Insecure DependenciesIntroduces malicious components through unsafe dependency sources
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
Findings (4)

T09 · Insecure Skill Coding Practices

Error
Location
SKILL.md:184
Finding

Unsafe Shell Command Templates Permit Command Injection

Content
View full analysis

Vulnerability Details

File Location: SKILL.md:184
Vulnerability Type: Unsafe construction of shell commands from target values
Risk Level: High

Vulnerable Code Snippet:

bash
npx tsc --noEmit target.ts

This command is one of multiple verification templates that place a target filename or path directly into a shell command. Other checks in the same document similarly instruct the Agent to substitute target paths into commands such as stat, grep, and language parsers.

Technical Analysis

The Skill does not define a safe argument-passing mechanism, canonical-path validation, or rejection of shell metacharacters. If an implementation replaces target.ts or analogous placeholders with an attacker-controlled target and executes the result through a shell, characters such as semicolons, command substitutions, redirections, or newlines can alter the command structure.

Quoting alone would not be sufficient if the implementation builds a command string incorrectly or invokes a shell with attacker-controlled content. The target must be passed as a discrete process argument.

Attack Path

  1. An attacker creates or identifies a target whose supplied path includes shell syntax.
  2. The attacker requests a hallucination check against that target.
  3. The Agent substitutes the supplied value into one of the documented shell templates.
  4. The generated string is passed to a shell.
  5. The shell interprets the injected syntax and executes the attacker's additional command.

This path depends on an Agent implementing the documented placeholders through shell-string interpolation rather than safe argument arrays.

Impact Assessment

Successful exploitation can execute arbitrary commands with the operating-system privileges of the Agent. Depending on the execution environment, this may allow reading accessible files and credentials, modifying the audited repository, initiating network requests, or c ...[truncated 51 chars]

Remediation
View remediation

Remediation Suggestions

  • Never concatenate a target path into a shell command string.
  • Invoke tools through structured process APIs with argument arrays.
  • Canonicalize targets and restrict them to the explicitly approved project root.
  • Reject paths containing control characters or unexpected shell metacharacters.
  • Insert -- before positional path arguments when the invoked tool supports it.
  • Quote shell variables consistently where shell use is unavoidable.
  • Run verification commands in a sandbox with minimal filesystem and network permissions.
  • Add regression tests using paths containing spaces, newlines, semicolons, command substitutions, and leading hyphens.

T09 · Insecure Skill Coding Practices

Error
Location
SKILL.md:667
Finding

Project-Controlled Configuration Can Define Arbitrary Shell Commands

Content
View full analysis

Vulnerability Details

File Location: SKILL.md:667-689
Vulnerability Type: Execution of unrestricted commands from repository configuration
Risk Level: Critical

Vulnerable Code Snippet:

yaml
custom_checks:
  - id: "H-C1"
    name: "Internal API endpoint verification"
    category: "fact"
    command: |
      curl -sI --max-time 5 https://internal-api.company.com/health | head -1
    pass_pattern: "200"
    severity: "error"
    
  - id: "H-C2"
    name: "Company terminology compliance"
    category: "consistency"
    command: |
      grep -n "end-user\|enduser\|end user" {target} | grep -v "# allowed"
    fail_pattern: "."
    severity: "warning"
    
  - id: "H-C3"
    name: "License header presence"
    category: "completeness"
    command: |
      head -3 {target} | grep -i "copyright\|license\|MIT\|Apache"
    pass_pattern: "."
    severity: "error"

Technical Analysis

The proposed .hallucination-guard.yaml format treats a repository-controlled command field as executable shell code. A repository is part of the audit input and must not be trusted to prescribe commands that run on the auditor's host.

No allowlist, command parser, signature requirement, user confirmation, sandbox boundary, or restriction on filesystem and network access is specified. The {target} substitution also creates an additional injection boundary if target values are inserted into these commands as text.

This turns a verification action into an arbitrary code-execution mechanism controlled by the project being inspected.

Attack Path

  1. An attacker adds .hallucination-guard.yaml to a repository.
  2. The configuration defines a custom check whose command performs a malicious action.
  3. A victim opens the repository and invokes the hallucination-guard Skill.
  4. The Agent discovers and accepts the repository-local configuration.
  5. The Agent executes the custom ...[truncated 709 chars]
Remediation
View remediation

Remediation Suggestions

  • Remove support for arbitrary command fields in repository-controlled configuration.
  • Replace shell commands with a declarative schema containing fixed check types and validated parameters.
  • Maintain a hardcoded allowlist of permitted tools and arguments.
  • Treat project-local configuration as untrusted and require explicit user approval before enabling it.
  • Display the exact proposed operation before execution.
  • Disable network access and restrict filesystem access for custom checks.
  • Execute checks in an isolated, disposable sandbox under an unprivileged account.
  • Validate {target} as a canonical path and pass it as a discrete argument rather than interpolating it into shell text.
  • If extensibility is required, use signed and administrator-installed plugins rather than repository-provided shell code.

T08 · Insecure Dependencies

Error
Location
SKILL.md:184
Finding

Unpinned Third-Party Package May Be Downloaded and Executed by npx

Content
View full analysis

Vulnerability Details

File Location: SKILL.md:184
Vulnerability Type: Unpinned and potentially implicit dependency retrieval
Risk Level: High

Vulnerable Code Snippet:

bash
npx tsc --noEmit target.ts

Technical Analysis

The command invokes npx without requiring a trusted, lockfile-pinned local TypeScript compiler and without using a no-install option. Depending on the installed npm/npx version and local project state, npx may retrieve a package when no suitable local executable exists.

The Skill does not specify an exact package version, integrity value, trusted registry, lockfile validation, or package lifecycle restrictions. Consequently, a syntax check can introduce externally sourced executable code into the audit process.

Attack Path

  1. The Agent is asked to validate a TypeScript target.
  2. A trusted local tsc executable is absent or not resolved.
  3. The Agent follows the documented npx tsc command.
  4. npx resolves or downloads a package from the configured registry.
  5. Package-controlled code executes with the Agent's privileges.
  6. A compromised registry account, unsafe registry configuration, or malicious package resolution can result in host compromise.

Impact Assessment

A malicious or compromised dependency can execute code under the Agent's account, access readable files and environment variables, modify the repository, and communicate over available network channels. The risk also includes nondeterministic audit behavior because the retrieved package can change independently of the reviewed Skill.

Remediation
View remediation

Remediation Suggestions

  • Require TypeScript to be installed as an exact, lockfile-pinned project dependency.
  • Invoke only the verified local binary.
  • Use npx --no-install tsc --noEmit target.ts when supported.
  • Fail safely or mark the check unavailable if the trusted local compiler is absent.
  • Enforce lockfile integrity with deterministic installation commands such as npm ci.
  • Configure an approved registry and validate package integrity metadata.
  • Disable dependency lifecycle scripts where they are not required.
  • Run compiler checks in a network-disabled sandbox with minimal privileges.

T05 · Unauthorized Access and Privilege Escalation

Error
Location
SKILL.md:826
Finding

Ambiguous Triggers Cause Automatic Broad-Scope Execution and Unapproved File Modification

Content
View full analysis

Vulnerability Details

File Location: SKILL.md:826-828
Vulnerability Type: Violation of least privilege and missing authorization for modifications
Risk Level: High

Vulnerable Code Snippet:

text
1. **On receiving "hallucination check" trigger** — Immediately start executing all 14 items
2. **When target is unclear** — Automatically set the current working file/directory as target
3. **On finding FAIL items** — Attempt to fix before reporting to user (state explicitly if fix is not possible)

Technical Analysis

The Skill instructs the Agent to execute checks immediately, silently expand an ambiguous request to the entire current working directory, and modify failed items before presenting the findings. Verification is normally a read-only operation, but these instructions authorize write behavior without identifying which files may change or requesting approval for each change.

Selecting the current working directory by default can also include a larger scope than the user intended. Combined with checks that classify TODO markers, placeholders, terminology, and unsupported assertions as failures, automatic remediation can alter legitimate source code, documentation, fixtures, or examples.

Attack Path

  1. A user or untrusted document causes the phrase hallucination check to be processed without an explicit target.
  2. The Skill selects the current working directory automatically.
  3. All checks run across that broader directory.
  4. Ordinary content is classified as failed, including intentional placeholders or incomplete-work markers.
  5. The Agent attempts to edit the affected files before reporting what it found.
  6. The user receives notice only after modifications have already been attempted.

Impact Assessment

The Agent may alter any writable file within the automatically selected working scope. Potential consequences include corrupting documentation, changing application beha ...[truncated 250 chars]

Remediation
View remediation

Remediation Suggestions

  • Make all verification read-only by default.
  • Require an explicit target and refuse to infer the entire working directory when the target is ambiguous.
  • Separate scanning from remediation into distinct phases.
  • Present proposed changes as a patch and obtain explicit user confirmation before writing files.
  • Require per-file authorization for changes outside the initially specified target.
  • Provide a dry-run mode and make it the default.
  • Preserve backups or use version-control-aware rollback for approved changes.
  • Exclude sensitive paths, repository metadata, dependencies, generated files, and files outside the canonical project root.
  • Report failures without automatically converting heuristic findings into source modifications.

SkillSpector

Vulnerability Patterns
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
Findings (6)

External Script Fetching

High
Category
Supply Chain
Confidence
90% confidence
Finding

Remote code is downloaded and executed. This bypasses code review and could introduce malicious code.

Content

Scanner excerpt · SKILL.md (reported line 598)May include surrounding context.

md
# stat → Test-Path / Get-Item
# which → Get-Command
# grep → Select-String
# curl → Invoke-WebRequest

# Wrapper function definitions
function hg-pathcheck { param($p) Test-Path $p }

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The skill explicitly recommends auto-running all 14 checks before completing any task, which includes filesystem inspection and optional network requests such as curl to external URLs and package registries. Triggering broad verification automatically without a clear user-facing consent step can expose local paths, installed tools, internal URLs, or metadata and may violate least-privilege expectations.

Content

No source excerpt is available for this finding.

Autonomous Decision Making

Medium
Category
Excessive Agency
Confidence
94% confidence
Finding

The skill instructs agents to 'automatically run' the checks before task completion, effectively delegating autonomous decisions about when to inspect the environment and potentially contact external services. In an agent setting, this reduces user control over side effects and can trigger sensitive actions without contextual approval.

Content

Scanner excerpt · SKILL.md (reported line 49)May include surrounding context.

text

### 2. Auto-Execution Method
Add to the system prompt so the agent automatically runs this skill before completing a task:

Before completing any task, you must run all 14 checks from the hallucination-guard SKILL.md,

Rp1

Medium
Category
MCP Rug Pull
Confidence
70% confidence
Finding

npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.

Content

No source excerpt is available for this finding.

External Transmission

Medium
Category
Data Exfiltration
Confidence
50% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · SKILL.md (reported line 755)May include surrounding context.

md
hallucination check on docs/api-reference.md --scope=fact,consistency

[Result]
H-3: ❌ https://api.example.com/v2/users → HTTP 404
     (Endpoint in documentation doesn't actually exist)
     
H-5: ❌ "Average response time 23ms" — no source

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
96% confidence
Finding

The instruction to automatically use the current working file or directory when the target is unclear can cause unintended scanning of sensitive project contents without explicit user direction. In a skill designed to run command-based checks, ambiguous targeting increases the chance of over-collection, privacy leakage, and accidental operation on unrelated files.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.