Back to skill

Security audit

Shipyard new-app

Security checks for vulnerabilities and agentic risk

Overview

This app-building skill needs review because it can remove access restrictions and add AI processing without clear privacy notices.

Review the full Shipyard plugin separately. Confirm which deployments should become publicly reachable before allowing protection changes; hiding an app from search engines does not restrict access. Require clear disclosure of which content AI providers receive. The Review verdict concerns these controls, not evidence of credential theft or malicious intent.

Vulnerability Patterns
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
Vulnerability Patterns
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Output HandlingUnvalidated Output Injection, Cross-Context Output, Unbounded Output
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
Findings (10)

Credential Access

High
Category
Privilege Escalation
Confidence
70% confidence
Finding

Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Content

Scanner excerpt · SKILL.md (reported line 24)May include surrounding context.

md
## Rules (always)

- You do the work. Run scripts and commands yourself. Never hand the student a command you could run.
- Never ask for keys in the chat and never read `~/.shipyard/secrets.env` or `.env*` files. Scripts handle keys and print only names with a tick or a cross.
- Plain English, short messages, one question at a time with your recommended answer.
- Verify before saying done. Say what was checked and what was not.
- Commits and pushes are the student's call: always ask "OK to save this to GitHub?" first. Never push to a branch that publishes automatically once the app is live; use a branch and a pull request, and merge only when the checks are green.

Credential Access

High
Category
Privilege Escalation
Confidence
70% confidence
Finding

Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Content

Scanner excerpt · references/api-and-mcp.md (reported line 8)May include surrounding context.

md
Build in this order, each step only when needed:

1. **A small JSON API** under `src/app/api/v1/`: list, read, create, update for the main item. Same rules as every route: check the user, rate limit, validate input (`src/lib/validate.mjs`), and only the person's own rows (row level security does the real check).
2. **Personal access tokens** (they expire: the person picks 30, 90 or 365 days or never, Settings shows "last used" and the expiry; rate limit token routes per IP and per token with 429 and `Retry-After`) so a script or agent can call the API without a browser: a `api_tokens` table (store only a hash of the token, show the token once, let the person name and revoke tokens on the Settings page). Add the route prefix to `BEARER_API_PREFIXES` in `src/proxy.ts` and check the token in the route. Never accept the Supabase secret key from outside.
3. **Webhooks out**: when something happens (an item is added or done), POST a small JSON body to a web address the person saves in Settin

Credential Access

High
Category
Privilege Escalation
Confidence
70% confidence
Finding

Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Content

Scanner excerpt · references/api-and-mcp.md (reported line 10)May include surrounding context.

md
IPv4-mapped IPv6 address, then connect to that checked IP (a pinned `lookup`, so a second DNS answer cannot swap it) with the TLS name still checked. No redirects, 10 s timeout, small response cap. Check when the address is saved and again on every send.
4. **An MCP server** so AI assistants can use the app as tools: a route at `src/app/api/mcp/route.ts` that speaks MCP over HTTP (the official `@modelcontextprotocol/sdk`), authenticated with the same personal tokens (or OAuth for directory-listed assistants: access tokens of about an hour, rotating refresh tokens, and a reused refresh token revokes the whole family). One tool per action the person would ask for ("list notes", "add a note", "mark done"), with short plain descriptions and input limits. Add a "Connect an AI assistant" box in Settings with the address and how to add it.

Every one of these gets tests (token hashing and revoking, signature of webhooks, each MCP tool refuses other people's data) and an entry in `HANDOFF.md`.

Credential Access

High
Category
Privilege Escalation
Confidence
70% confidence
Finding

Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Content

Scanner excerpt · references/lessons.md (reported line 58)May include surrounding context.

md
event-id table has RLS on and no grants for the browser roles. (The billing module does all this.)
- Webhooks out to an address the person typed: https only, port 443, resolve the name and refuse if any address is private (127/8, 10/8, 172.16/12, 192.168/16, 169.254/16, 100.64/10, ::1, fc00::/7, IPv4-mapped IPv6), connect to the checked IP (no second lookup), no redirects, a timeout and a response size cap. Check at save time and again at send time.
- Tokens for agents and scripts expire: short access tokens with rotating refresh tokens (a reused refresh token revokes the whole family), personal tokens with a chosen lifetime and "last used", and per-IP / per-token rate limits answering 429 with `Retry-After`.
- Analytics (PostHog, GA) starts only after the visitor accepts it; withdrawing is one click in Settings and the footer and stops capture at once.
- A paid plan whose payments are not set up yet shows a disabled "Coming soon" button, never a checkout that cannot finish.
- Spend li

Credential Access

High
Category
Privilege Escalation
Confidence
70% confidence
Finding

Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Content

Scanner excerpt · references/lessons.md (reported line 18)May include surrounding context.

md
erences/rls-speed.mjs` checks a live project.
- Views ignore RLS unless `with (security_invoker = true)`; a policy without `to authenticated` applies to visitors who are not signed in; security definer functions must be revoked from public, anon and authenticated. The app's `schema-rules` test checks all three.
- A key deleted in the next commit is still in the history that gets pushed. The pre-push hook and CI scan every pushed commit, not just the files on disk.
- Account-level keys (Supabase access token) never go into an app's hosting settings.
- ★ No secrets in `NEXT_PUBLIC_*` variables. `.env*` stays ignored by git. Scan for keys before every push.
- ★ A one-time token is never used on a GET. Mail scanners open every link in an email and use the token up (the person sees "link expired"), and a crafted link can sign a visitor into the sender's account. The link opens a page with one button, and the button POSTs. `/auth/confirm` does this.
- No change on a GET at all (sign-out, del

Credential Access

High
Category
Privilege Escalation
Confidence
70% confidence
Finding

Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Content

Scanner excerpt · references/shared-project.md (reported line 13)May include surrounding context.

md
## Steps

1. Find the project id: Supabase dashboard, the project, the address contains it (20 letters and numbers). Never ask for the access token in chat.
2. Dry run, change nothing: `node "${CLAUDE_PLUGIN_ROOT}/scripts/provision-supabase.mjs" <appDir> --shared <projectId> --dry-run`. It saves how the project looks now, then prints the database steps rewritten for the app's schema, the new API list and the redirect addresses it would add. Read it to the student in plain words (new space, nothing else changes). If it refuses a database file, fix that file (see below) and run the dry run again.
3. Real run: the same command without `--dry-run`. Add `--schema <name>` to pick the space name (letters, digits, underscores; the default is fine).
4. It checks afterwards that nothing else in the project changed (sign-in main address, email settings, the public schema, the sign-up trigger, the list of schemas the API serves for other apps). If it stops with "unexpected change", show the stude

Instructions to add AI features omit a user-facing warning that input data will be sent to third-party providers.

Medium
Category
Output Handling
Confidence
94% confidence
Finding

The instructions encourage processing messages, photos, and speech through external AI providers without requiring disclosure of that transfer or controls for sensitive data. Server-side keys, authentication, and rate limits do not prevent private user content from being disclosed to a third party without the user's informed understanding.

Content

No source excerpt is available for this finding.

Autonomous Decision Making

Medium
Category
Excessive Agency
Confidence
75% confidence
Finding

Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.

Content

Scanner excerpt · references/shared-project.md (reported line 16)May include surrounding context.

md
se changes). If it refuses a database file, fix that file (see below) and run the dry run again.
3. Real run: the same command without `--dry-run`. Add `--schema <name>` to pick the space name (letters, digits, underscores; the default is fine).
4. It checks afterwards that nothing else in the project changed (sign-in main address, email settings, the public schema, the sign-up trigger, the list of schemas the API serves for other apps). If it stops with "unexpected change", show the student the commands it printed to put things back, and do not run them without asking. Nothing is reverted automatically.
5. It writes `NEXT_PUBLIC_SUPABASE_SCHEMA` into `.env.local` and records `schema` and `shared` in `.shipyard-app.json`. `provision-vercel.mjs` then sends the schema name to Vercel too. Run the app's `verify`: it sends the schema on every database call.

Adding a module later: run `provision-supabase.mjs <appDir>` again (no flag needed). A shared app always goes through the shared flow.

The remediation instructs disabling Vercel Deployment Protection without warning that this removes an access restriction.

Medium
Category
Output Handling
Confidence
91% confidence
Finding

The failure-remediation column explicitly instructs the agent to turn off Deployment Protection, rather than merely listing an available tool. It does not warn that previously restricted deployment content could become publicly accessible or establish that the owner intends public access.

Content

No source excerpt is available for this finding.

MCP server referenced without pinned version: 'npx playwright'.

Medium
Category
MCP Rug Pull
Confidence
70% confidence
Finding

npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.

Content

No source excerpt is available for this finding.