Back to skill

Security audit

Karpathy Wiki Improve

Security checks for vulnerabilities and agentic risk

Overview

The skill is a coherent wiki organizer, but it needs Review because broad triggers can lead to persistent local wiki edits, web searches, and periodic maintenance without enough scoping or consent.

Install only if you are comfortable with a skill that can maintain a persistent local wiki. Use a dedicated, backed-up wiki folder without unrelated private files, approve web-search queries and file changes before execution, avoid periodic maintenance unless explicitly enabled, and prefer a pinned or otherwise verified installer command.

Vulnerability Patterns
  • Insecure DependenciesIntroduces malicious components through unsafe dependency sources
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (1)

T08 · Insecure Dependencies

Warning
Location
README.md:16
Finding
Unpinned Package Execution Through npx## Vulnerability Details **File Location**: `README.md:16` **Vulnerability Type**: Unpinned third-party package execution **Risk Level**: Medium ### Vulnerable Code ```bash npx clawhub install zhangmengyang/karpathy-wiki-improve ``` ### Technical Analysis The documented installation command invokes the `clawhub` npm package through `npx` without specifying a reviewed version or integrity constraint. If the package is not already available locally, `npx` can retrieve and execute the version currently resolved by the configured npm registry. This creates a supply-chain risk because the code executed by the installation command can change after this skill has been reviewed. A compromised maintainer account, malicious package release, registry compromise, or unexpected upstream update could cause users following the documentation to execute attacker-controlled code. The repository itself does not contain an embedded malicious script, and the audit found no evidence that the referenced package is currently malicious. The issue is the unsafe, unpinned execution mechanism. ### Attack Path 1. An attacker compromises the publication channel, maintainer account, or resolved package for the `clawhub` CLI. 2. The attacker publishes a malicious or backdoored version that remains compatible with the documented command. 3. A user follows the Quick Start instructions and runs `npx clawhub install zhangmengyang/karpathy-wiki-improve`. 4. `npx` resolves and downloads the compromised version because no package version or integrity value is specified. 5. The downloaded package executes with the privileges of the invoking user. 6. Malicious installation logic can access or modify data available to that user, subject to operating-system and sandbox restrictions. ### Impact Assessment Successful exploitation could provide arbitrary code execution under the account running the installation command. The potential scope includes readable ...[truncated 395 chars]
Remediation
## Remediation Suggestions 1. Pin the CLI to a specific, reviewed version: ```bash npx clawhub@<reviewed-version> install zhangmengyang/karpathy-wiki-improve ``` 2. Document the expected npm registry and verified package identity so that scoped registry overrides or package-name confusion cannot silently alter the source. 3. Where supported, verify package provenance, signatures, checksums, or npm integrity metadata before execution. 4. Test and review each new CLI version before updating the documented pin. 5. Recommend running the installer as an unprivileged user in a restricted environment without unnecessary secrets. 6. For stronger reproducibility, install the reviewed dependency through a lockfile with integrity metadata and invoke the locally locked binary rather than resolving the latest package dynamically.
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
Findings (8)

Missing User Warnings

Medium
Confidence
96% confidence
Finding
The README advertises persistent ingestion, relinking, linting, and deep-research workflows that modify a markdown wiki, but it does not clearly warn users that these actions write to disk and maintain a persistent knowledge graph. Users may invoke the skill expecting analysis-only behavior and inadvertently cause durable changes to local content or source archives. In a system handling bookmarks, research notes, and source traceability, silent persistence increases the chance of privacy, integrity, and workflow-impact issues.

Rp1

Medium
Category
MCP Rug Pull
Confidence
92% confidence
Finding
The README instructs users to run `npx clawhub install zhangmengyang/karpathy-wiki-improve` without pinning a specific version. This creates a supply-chain risk because users may fetch whatever package version is current at execution time, including a compromised or malicious update. In a skill that can ingest, research, relink, and modify a persistent wiki, that risk is more concerning because installed code may gain access to local knowledge data and perform writes.

Vague Triggers

Medium
Confidence
94% confidence
Finding
The trigger phrases like `research X`, `relink`, and `organize my bookmarks` are broad and overlap with ordinary user intent, which can cause the skill to activate unexpectedly. Because this skill performs persistent ingest and wiki modification, accidental invocation could lead to unintended writes, web access, or restructuring of a user's knowledge base. The operational context makes this more dangerous than a read-only skill.

Vague Triggers

Medium
Confidence
94% confidence
Finding
The activation phrases are broad enough to match ordinary requests like 'research X', 'wiki', or 'help me research', which can cause the skill to activate outside the user's clear intent. Because this skill can read, write, relink, lint, and perform external fetching/search as part of its workflow, accidental activation increases the chance of unintended file modification and network access.

Missing User Warnings

Medium
Confidence
91% confidence
Finding
The operations section describes creating/updating pages, syncing backlinks, updating index.md, and appending to log.md, but it does not prominently warn users that these actions modify local files automatically. Users may invoke the skill expecting analysis or advice, while the skill is actually designed to persist changes across a local wiki tree.

Missing User Warnings

Medium
Confidence
92% confidence
Finding
The Deep Research workflow instructs the agent to run external web searches, collect results, store them under raw/sources, and then ingest them, but it lacks a conspicuous warning about both outbound network activity and local persistence of fetched content. This creates privacy and consent risks, especially if user topics, internal interests, or sensitive research areas are sent to third-party search providers and saved locally.

Vague Triggers

Medium
Confidence
87% confidence
Finding
The 'periodic heartbeat triggers lint + relink + quality check' use case implies autonomous or time-based activation without an explicit per-run user request. In this context, that is risky because relink and lint can lead to local file inspection and updates, making the skill behave more like a background agent than a user-invoked tool.

Natural-Language Policy Violations

Low
Confidence
84% confidence
Finding
The rule "No mixed Chinese/English" imposes a language constraint on content naming without any opt-in, locale choice, or documented regional justification. This is a natural-language policy concern because it forces a language/locale convention rather than presenting it as a configurable option.

Static analysis

No suspicious patterns detected.