Back to skill

Security audit

Shipyard new-skill

Security checks for vulnerabilities and agentic risk

Overview

This skill coherently guides skill creation, with explicit approval for private-content access, installation, building, and publishing.

This requires the full Shipyard plugin, whose scripts were not inspected here. Review that plugin and optional third-party skills before installation, and prefer fixed installer versions. Limit approved code and chat access to what the task needs.

Vulnerability Patterns
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
Vulnerability Patterns
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
Findings (5)

Credential Access

High
Category
Privilege Escalation
Confidence
70% confidence
Finding

Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Content

Scanner excerpt · SKILL.md (reported line 16)May include surrounding context.

md
- You do the work. Run scripts yourself. Never hand the student a command you could run.
- Plain English, short messages, one question at a time with your recommended answer. Explain any new word in five words or fewer ("front matter: the name and description block").
- Never read `~/.shipyard/secrets.env` or `.env*` files, and never put a key in a skill. A skill that needs a key reads it from a file or variable the student fills in themselves.
- Read the student's code and agent chats only with a yes, and only the parts you need: file lists, READMEs, the instruction files (`AGENTS.md`, `CLAUDE.md`, `.cursor/rules`), and summaries of chats, not whole transcripts. Say what you read.
- Publishing, pushing, opening pull requests on other people's repos and filling in submission forms: ask first, wait for a clear yes, once per action.
- Verify before saying done. Say what was checked and what was not.

Autonomous Decision Making

Medium
Category
Excessive Agency
Confidence
75% confidence
Finding

Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.

Content

Scanner excerpt · SKILL.md (reported line 32)May include surrounding context.

md
## 2. Push back, then let them choose

Read `references/pushback.md` and check the spec against it. If something is weak (too broad, duplicates a skill they already have, needs keys in the chat, publishes without asking, could be a one-line rule instead), say so in one or two sentences, give your better version, and ask which they prefer. Their choice wins unless it is unsafe; then say plainly why you will not build that part.

## 3. Recommend where to build it

Autonomous Decision Making

Medium
Category
Excessive Agency
Confidence
75% confidence
Finding

Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.

Content

Scanner excerpt · references/pushback.md (reported line 10)May include surrounding context.

md
- **Already exists.** If Shipyard or a skill they have does it, offer to extend that one. Two skills with the same triggers fight.
- **No trigger words.** If they cannot say what they would type, the agent will not know when to load it. Find the words first.
- **Keys in the chat or in the skill.** Never. The key goes in a file the student fills in; scripts read it and print only a tick.
- **Acts without asking.** Publishing, sending, paying, deleting and merging always ask first. A skill that does them silently is refused, even if they insist.
- **Judgement in prose that a script could check.** If a step is "make sure X", write a script or a test that checks X. Prose drifts; checks do not.
- **Locked to one agent for no reason.** Claude Code only features (hooks, slash commands) are fine, but say what Codex and Cursor users get instead.
- **Huge.** A SKILL.md over about 100 lines gets skimmed. Move detail into `references/` and load it only at the step that needs it.

MCP server referenced without pinned version: 'npx skills'.

Medium
Category
MCP Rug Pull
Confidence
93% confidence
Finding

Although this is an npm CLI rather than an MCP server, npx skills can download and execute an unpinned package version with the invoking user's permissions. A compromised future release could execute malicious code; user consent and reading the target skill do not verify the installer package itself.

Content

No source excerpt is available for this finding.

MCP server referenced without pinned version: 'npx skills'.

Medium
Category
MCP Rug Pull
Confidence
88% confidence
Finding

The publishing guide recommends an installation command that executes an unpinned npm CLI, rather than referencing an MCP server. Users following these instructions could run a compromised future package release; presenting it as documentation reduces immediate exposure but still propagates an unsafe installation pattern.

Content

No source excerpt is available for this finding.