Back to skill

Security audit

Security Audit Tools

Security checks for vulnerabilities and agentic risk

Overview

The skill is a security-audit helper, but it overstates its coverage and includes unsafe audit scripts that can fetch or execute untrusted package-related content without enough validation.

Review this skill carefully before installing. Its goal is useful, but do not treat its LOW RISK or PASS outputs as installation approval. Run its helper scripts only in an isolated workspace with no credentials, prefer read-only/manual review first, and avoid the npm audit script until package names, versions, tarball paths, and repository URLs are validated and quoted.

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (4)

T09 · Insecure Skill Coding Practices

Error
Location
tools/audit-npm-package.sh:46
Finding

Attacker-Controlled Repository Metadata Can Inject Git Command-Line Arguments

Content
View full analysis
&1 | grep -E "Cloning|Receiving" || echo "Repository clone failed or not needed" else echo "⚠️ No GitHub repository found in package metadata" fi ``` ### Technical Analysis The repository URL originates from npm registry metadata controlled by the package publisher. The value is expanded without quotes in the `git clone` command: ```bash git clone --depth 50 $REPO_URL github-repo ``` Unquoted shell expansion performs word splitting and pathname expansion. A malicious metadata value containing whitespace can therefore become multiple Git arguments rather than one repository URL. Tokens beginning with `-` may be interpreted as Git options. Git clone supports options and configuration values that influence transports and protocol handling. Consequently, crafted metadata could alter clone behavior, select an unintended transport, or attempt to activate a dangerous Git protocol or helper. The precise impact depends on the installed Git version, enabled protocols, and local configuration, but package metadata must not be allowed to control Git options. The script also does not restrict repositories to trusted HTTPS hosts. This exceeds the minimum network privileges required for source comparison because the audit may connect to an arbitrary host or transport selected by an untrusted package publisher. ### Attack Path 1. An attacker publishes an npm package ...[truncated 1496 chars]
Remediation
View remediation
&2 exit 1 ;; esac case "$REPO_URL" in *[[:space:][:cntrl:]]*) echo "Rejected malformed repository URL" >&2 exit 1 ;; esac ``` 3. Disable dangerous protocols explicitly where supported: ```bash git \ -c protocol.file.allow=never \ -c protocol.ext.allow=never \ clone --depth 50 --end-of-options "$REPO_URL" github-repo ``` 4. Avoid inheriting credentials for arbitrary hosts. Run Git with an isolated configuration and disabled credential helper when authentication is unnecessary. 5. Treat absent, malformed, or non-allowlisted repository metadata as a manual-review condition rather than attempting the clone. ]]>

T09 · Insecure Skill Coding Practices

Warning
Location
tools/audit-npm-package.sh:8
Finding

Unvalidated Package and Version Inputs Permit npm Option Injection and Unsafe Output Paths

Content
View full analysis
" echo "Example: $0 @catalyst-team/poly-sdk 0.5.0" exit 1 fi PACKAGE_SAFE=$(echo $PACKAGE | tr '/' '-') OUTPUT_DIR="audit-${PACKAGE_SAFE}-${VERSION}" ``` ```bash # 1. Get NPM metadata echo "[1/6] Fetching NPM metadata..." npm view $PACKAGE@$VERSION --json > npm-metadata.json ``` ```bash # 2. Download NPM package echo "" echo "[2/6] Downloading NPM package..." npm pack $PACKAGE@$VERSION ``` ### Technical Analysis `PACKAGE` and `VERSION` are accepted without syntactic validation. They are then expanded without quotes in commands that invoke npm: ```bash npm view $PACKAGE@$VERSION --json npm pack $PACKAGE@$VERSION ``` Unquoted expansion permits shell word splitting and pathname expansion. Input containing whitespace can therefore produce additional npm arguments. Because option parsing is not terminated with `--`, attacker-supplied tokens beginning with `-` may be interpreted as npm options rather than as part of the requested package specification. The output-directory derivation is also unsafe. Only `/` characters in `PACKAGE` are replaced, while `VERSION` remains unsanitized: ```bash OUTPUT_DIR="audit-${PACKAGE_SAFE}-${VERSION}" ``` Although later directory operations quote `"$OUTPUT_DIR"`, an attacker-controlled version can still contain traversal components or other path-significant content. Quoting prevents word splitting but does not make an unsafe path safe. ### Attack Path 1. A caller supplies a crafted package or version argument containing whitespace, npm options, glob characters, or path traversal sequences. 2. The script constructs `$PACKAGE@$VERSION` without validation. ...[truncated 1205 chars]
Remediation
View remediation
npm-metadata.json npm pack -- "$SPEC" ``` The exact placement of `--` should be verified against the supported npm CLI versions. 5. Do not derive filesystem paths directly from raw input. Generate a safe identifier or use a temporary directory: ```bash OUTPUT_DIR=$(mktemp -d "${TMPDIR:-/tmp}/npm-audit.XXXXXXXX") ``` 6. Pin the registry or allowlist approved registry origins rather than permitting caller-controlled npm configuration to redirect downloads. 7. Record the resolved package name, version, tarball URL, and integrity value and verify that they match the requested immutable specification. ]]>

T09 · Insecure Skill Coding Practices

Warning
Location
tools/quick_scan.py:25
Finding

Incomplete Pattern Scanner Produces an Unsafe “Safe to Install” Verdict

Content
View full analysis
= 70: print("\n❌ HIGH RISK - Do not install") sys.exit(1) elif report['risk_score'] >= 40: print("\n⚠️ MEDIUM RISK - Manual review required") sys.exit(0) else: print("\n✅ LOW RISK - Safe to install") sys.exit(0) ``` The unsafe assurance is reinforced in `README.md:89-91`: ```text * **0-39**: ✅ LOW RISK (Safe to install) * **40-69**: ⚠️ MEDIUM RISK (Manual review required) * **70-100**: ❌ HIGH RISK (Installation prohibited) ``` ### Technical Analysis The scanner inspects only files matching four JavaScript and TypeScript extensions: - `.ts` - `.js` - `.tsx` - `.jsx` It explicitly skips test directories and does not inspect many locations capable of carrying supply-chain payloads, including: - Shell and PowerShell scripts. - Python, Ruby, or other executable source. - `package.json` lifecycle hooks. - GitHub Actions workflows. - Installer configuration. - Native binaries and compiled modules. - Extensionless executable files. - Files deliberately placed under skipped directories. - Generated or encoded assets loaded at runtime. Within selected files, detection is limited to ten regular-expression patterns applied one line at a time. This approach does not establish data flow, resolve aliases, identify split strings, inspect indirect calls, or detect many semantically equivalent malicious implementations. Despite these limitations, a score below 40 is present ...[truncated 1842 chars]
Remediation
View remediation

T09 · Insecure Skill Coding Practices

Warning
Location
tools/audit-git-history.sh:53
Finding

Git History Checks Can Suppress Failures and Falsely Report Successful Analysis

Content
View full analysis
Remediation
View remediation
&2 exit 1 fi matches=$(printf '%s\n' "$history" | grep -E '^\+.*\beval\b|^\+.*new Function\(' || true) if [ -n "$matches" ]; then printf '%s\n' "$matches" | head -10 else echo "No eval/Function() additions found" fi ``` 3. Distinguish these states in both output and exit codes: - Scan completed with no matches. - Matches found and manual review required. - Scan failed or was incomplete. 4. Avoid substring-based domain exclusions. Parse and validate complete URL hostnames instead. 5. Return a nonzero exit status whenever history extraction or any required analysis stage fails. 6. Include repository completeness information, such as shallow-clone status, available refs, commit count, and whether all intended history was scanned. ]]>
Vulnerability Patterns
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
Findings (30)

Tp4

High
Category
MCP Tool Poisoning
Confidence
97% confidence
Finding

The declared scope includes reputation checks, pre-download inspection, post-download deep inspection, and broad supply-chain threat detection, but the actual described logic is largely superficial and local-file oriented. Such gaps create a material risk of missed malicious behavior while the user believes a professional-grade audit occurred.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding

The declared scope includes reputation checks, pre-download inspection, post-download deep inspection, and broad supply-chain threat detection, but the actual described logic is largely superficial and local-file oriented. Such gaps create a material risk of missed malicious behavior while the user believes a professional-grade audit occurred.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
97% confidence
Finding

The declared scope includes reputation checks, pre-download inspection, post-download deep inspection, and broad supply-chain threat detection, but the actual described logic is largely superficial and local-file oriented. Such gaps create a material risk of missed malicious behavior while the user believes a professional-grade audit occurred.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
97% confidence
Finding

The declared scope includes reputation checks, pre-download inspection, post-download deep inspection, and broad supply-chain threat detection, but the actual described logic is largely superficial and local-file oriented. Such gaps create a material risk of missed malicious behavior while the user believes a professional-grade audit occurred.

Content

No source excerpt is available for this finding.

External Script Fetching

High
Category
Supply Chain
Confidence
90% confidence
Finding

Remote code is downloaded and executed. This bypasses code review and could introduce malicious code.

Content

Scanner excerpt · SKILL.md (reported line 214)May include surrounding context.

md
1. **Never install first and review later.**
2. **Never execute installer scripts during review.**
3. **Never run `curl ... | sh`, `wget ... | bash` during review.**
4. **Never夸大审计能力。** 明确告知用户实际做了什么。
5. **Always深入分析关键源文件。** grep不是分析。
6. **Always量化审计深度。** 报告审查了多少行代码。

External Script Fetching

High
Category
Supply Chain
Confidence
90% confidence
Finding

Remote code is downloaded and executed. This bypasses code review and could introduce malicious code.

Content

Scanner excerpt · SKILL.md (reported line 214)May include surrounding context.

md
1. **Never install first and review later.**
2. **Never execute installer scripts during review.**
3. **Never run `curl ... | sh`, `wget ... | bash` during review.**
4. **Never夸大审计能力。** 明确告知用户实际做了什么。
5. **Always深入分析关键源文件。** grep不是分析。
6. **Always量化审计深度。** 报告审查了多少行代码。

Credential Access

High
Category
Privilege Escalation
Confidence
70% confidence
Finding

Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Content

Scanner excerpt · findings-catalog.md (reported line 6)May include surrounding context.

md
## High-confidence malicious patterns

### Secret theft
- Reads `.env`, `.npmrc`, `.pypirc`, `.git-credentials`, `.ssh`, cloud credential files, browser cookies, keychain/keyring stores
- Enumerates env vars and posts them outbound
- Scans home directory for API keys or wallet files

Credential Access

High
Category
Privilege Escalation
Confidence
90% confidence
Finding

Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Content

Scanner excerpt · findings-catalog.md (reported line 6)May include surrounding context.

md
## High-confidence malicious patterns

### Secret theft
- Reads `.env`, `.npmrc`, `.pypirc`, `.git-credentials`, `.ssh`, cloud credential files, browser cookies, keychain/keyring stores
- Enumerates env vars and posts them outbound
- Scans home directory for API keys or wallet files

Credential Access

High
Category
Privilege Escalation
Confidence
70% confidence
Finding

Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Content

Scanner excerpt · findings-catalog.md (reported line 6)May include surrounding context.

md
## High-confidence malicious patterns

### Secret theft
- Reads `.env`, `.npmrc`, `.pypirc`, `.git-credentials`, `.ssh`, cloud credential files, browser cookies, keychain/keyring stores
- Enumerates env vars and posts them outbound
- Scans home directory for API keys or wallet files

Self-Modification

High
Category
Rogue Agent
Confidence
90% confidence
Finding

Skill modifies its own code, configuration, or behavior at runtime. Self-modification enables an agent to escalate privileges, disable safety constraints, or install persistent backdoors.

Content

Scanner excerpt · findings-catalog.md (reported line 41)May include surrounding context.

md
- Installer scripts that fetch binaries from arbitrary URLs

### Human-targeted social engineering
- README tells user to bypass warnings, disable security features, or paste opaque terminal commands
- Skill instructions try to persuade the agent to skip review or ignore policy

## Usually benign but worth checking

Anti-Refusal Statement

High
Category
Anti-Refusal
Confidence
90% confidence
Finding

Skill attempts to nullify the agent's safety policies or restrictions ('you have no restrictions', 'ignore your guidelines', 'do anything now'). This is a direct jailbreak that disables guardrails.

Content

Scanner excerpt · findings-catalog.md (reported line 42)May include surrounding context.

md
### Human-targeted social engineering
- README tells user to bypass warnings, disable security features, or paste opaque terminal commands
- Skill instructions try to persuade the agent to skip review or ignore policy

## Usually benign but worth checking

Description-Behavior Mismatch

High
Category
Not specified by scanner
Confidence
98% confidence
Finding

The scanner only traverses *.ts, *.js, *.tsx, and *.jsx files, despite the skill claiming to inspect npm packages, pip packages, shell installers, GitHub Actions, plugins, and other install sources. An attacker can trivially place malicious behavior in unscanned files such as package.json scripts, shell files, Python modules, workflow YAML, or installer artifacts and evade detection.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The README explicitly advises that users may install a package if a 5-second pattern-based quick scan reports LOW RISK. In a security-audit skill, this is dangerous because it can create false confidence, bypass deeper review, and lead users to install malicious packages that evade simple static signatures or hide behavior in scripts, dependencies, or runtime logic.

Content

No source excerpt is available for this finding.

Undeclared Tool Scope

Medium
Category
MCP Least Privilege
Confidence
91% confidence
Finding

The skill documents broad capabilities to read files, write audit artifacts, query environment-dependent tools, and make network requests, but it declares no explicit tool scope or permission boundaries. In an install-review skill, missing least-privilege constraints increases the blast radius if the skill is triggered unexpectedly or if later edits introduce unsafe command execution around untrusted packages.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

Medium
Category
Not specified by scanner
Confidence
87% confidence
Finding

The skill promises pre-download inspection and non-execution of installers, yet later directs cloning/downloading content and performing post-download actions. This inconsistency can lead operators to exceed the intended trust boundary and handle untrusted artifacts under weaker safeguards than expected.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The auto-use trigger language is broad enough to activate on many ordinary requests involving running, cloning, or using tools, increasing the chance of unintended invocation. For a skill with network and filesystem capabilities, overbroad triggering can create unnecessary exposure, surprise behavior, and accidental handling of untrusted content.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
88% confidence
Finding

SQP-3 applies to all file types and covers language or locale policy issues. Large portions of the instructions, examples, and required reporting text are Chinese-only, which can effectively force a language choice on users or downstream outputs without opt-in.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Medium
Category
Not specified by scanner
Confidence
88% confidence
Finding

The documentation says not to execute installer scripts and emphasizes read/review behavior, but later instructs loading the package in Node, which is still code execution. In the context of inspecting third-party packages, this inconsistency can cause analysts to run attacker-controlled module initialization code while believing they are staying in a safe review mode.

Content

No source excerpt is available for this finding.

External Transmission

Medium
Category
Data Exfiltration
Confidence
50% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · SKILL.md (reported line 309)May include surrounding context.

  1. GitHub仓库分析
    bash
    # 使用API获取完整元数据
    curl -s https://api.github.com/repos/owner/repo | jq '{...}'
    
    # 检查Issues(安全相关)
    curl -s "https://api.github.com/repos/owner/repo/issues?labels=security"
    

External Transmission

Medium
Category
Data Exfiltration
Confidence
50% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · SKILL.md (reported line 312)May include surrounding context.

  1. GitHub仓库分析
    bash
    # 使用API获取完整元数据
    curl -s https://api.github.com/repos/owner/repo | jq '{...}'
    
    # 检查Issues(安全相关)
    curl -s "https://api.github.com/repos/owner/repo/issues?labels=security"
    

External Transmission

Medium
Category
Data Exfiltration
Confidence
50% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · SKILL.md (reported line 315)May include surrounding context.

  1. GitHub仓库分析
    bash
    # 使用API获取完整元数据
    curl -s https://api.github.com/repos/owner/repo | jq '{...}'
    
    # 检查Issues(安全相关)
    curl -s "https://api.github.com/repos/owner/repo/issues?labels=security"
    

Context-Inappropriate Capability

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

The workflow authorizes executing downloaded package code in a sandbox (node -e "require('package')"), which conflicts with the stated non-executing review posture. Even in a sandbox, executing untrusted code expands the attack surface and may violate operator expectations for a review-only skill.

Content

No source excerpt is available for this finding.

Rp1

Medium
Category
MCP Rug Pull
Confidence
83% confidence
Finding

The documented Docker invocation uses an unpinned base image (node:20 implied later) without a digest, so analysis may run in a drifting environment and silently consume a compromised or changed image. For a security-audit skill, relying on mutable analysis infrastructure weakens trust in results and can expose the host or inspected code to supply-chain risk.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

The keyword-based auto-trigger section is ambiguous and can match generic coding or review discussions, not just pre-install security checks. In this skill context, accidental activation matters because the workflow may initiate external lookups, package downloads, or persistence beyond what the user intended.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

The script immediately performs npm pack and may git clone a repository as part of an audit flow, which conflicts with the stated security goal of inspecting third-party artifacts before any download or installation. Although it does not execute install scripts, it still causes network retrieval of untrusted content and can expose the operator to unnecessary risk, especially in a tool that users may trust as a safe pre-install gate.

Content

No source excerpt is available for this finding.

Static analysis

Detected: suspicious.dangerous_exec, suspicious.env_credential_access

Shell command execution detected (child_process).

Critical
Code
suspicious.dangerous_exec
Location
tools/audit-source-code.sh:61

Environment variable access combined with network send.

Critical
Code
suspicious.env_credential_access
Location
tools/quick_scan.py:73