Back to skill

Security audit

setlist

Security checks for vulnerabilities and agentic risk

Overview

This skill is mostly a setlist.fm MCP wrapper, but it needs review because its read-only framing conflicts with account-changing attendance tools and its setup runs unpinned external code.

Review before installing. Pin the setlist-mcp package or a specific audited commit, keep SETLIST_API_KEY scoped and private, and treat attendance tools as account-changing actions despite the read-only wording elsewhere in the documentation.

Vulnerability Patterns
  • Insecure DependenciesIntroduces malicious components through unsafe dependency sources
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (1)

T08 · Insecure Dependencies

Warning
Location
SKILL.md:17
Finding
Unpinned Third-Party Code Is Downloaded and Executed<![CDATA[ ## Vulnerability Details **File Location**: `SKILL.md`, lines 17–36 **Vulnerability Type**: Unpinned executable dependency and mutable remote source **Risk Level**: Medium ### Vulnerable Code ```markdown Add to `.mcp.json` in your project or `~/.claude/mcp.json`: ```json { "mcpServers": { "setlist": { "command": "npx", "args": ["-y", "setlist-mcp"], "env": { "SETLIST_API_KEY": "your-api-key-here" } } } } ``` ### Option B — from source ```bash git clone https://github.com/chrischall/setlist-mcp cd setlist-mcp npm install && npm run build ``` ``` ### Technical Analysis The recommended configuration invokes `npx -y setlist-mcp` without specifying an audited version. The `-y` option suppresses interactive confirmation, allowing npm to download and execute whichever package version currently satisfies the request. Consequently, the code executed later may differ from the code that was reviewed when this Skill was published. The alternative installation method has the same mutability issue: it clones the repository's default branch without pinning a commit or signed release and then executes package lifecycle and build scripts through `npm install && npm run build`. Installation scripts belonging to the package or any transitive dependency can execute with the privileges of the installing user. The referenced npm package and GitHub repository are consistent with the Skill's declared functionality, and the audit found no evidence that either is currently malicious. The vulnerability is the absence of integrity and version controls around remotely retrieved executable code. The MCP process is deliberately provided with `SETLIST_API_KEY`. If an upstream package, repository, maintainer account, or transitive dependency were compromised, downloaded code could read that credential and access any other files, environment variables, and network resources available to the MCP host process. ### Attack Path 1. A ...[truncated 1566 chars]
Remediation
<![CDATA[ ## Remediation Suggestions 1. Pin the npm package to a specifically audited version: ```json { "command": "npx", "args": ["-y", "setlist-mcp@<audited-version>"] } ``` 2. Prefer a locally installed dependency governed by a committed lockfile rather than downloading executable code whenever the MCP server starts. 3. For source installations, check out an audited immutable commit or signed release: ```bash git clone https://github.com/chrischall/setlist-mcp cd setlist-mcp git checkout --detach <audited-commit-sha> npm ci npm run build ``` 4. Verify package provenance, release signatures, and integrity hashes before installation. Review npm lifecycle scripts and the transitive dependency tree. 5. Update dependencies through a controlled review process rather than automatically tracking the latest package or default repository branch. 6. Run the MCP server under a dedicated, least-privileged account or sandbox with access only to required files and network destinations. 7. Supply `SETLIST_API_KEY` through a protected secret-management mechanism where supported. Restrict permissions on `.mcp.json` or `.env` if those files must contain the key. 8. Limit outbound network access to the setlist.fm API and other destinations strictly required for package installation or operation. ]]>
Vulnerability Patterns
  • Rogue AgentSelf-Modification, Session Persistence
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
Findings (5)

MCP Config Access

High
Category
Agent Snooping
Content
### Option A — npx (recommended)

Add to `.mcp.json` in your project or `~/.claude/mcp.json`:

```json
{
Confidence
95% confidence
Finding
Skill accesses MCP server configuration files (mcp.json). MCP configs contain server URLs, authentication tokens, and tool definitions — reading them allows the skill to discover and potentially abuse other tool integrations.

Vague Triggers

Medium
Confidence
93% confidence
Finding
The manifest says the skill triggers on "any request about concert setlists, gigs, tours, or live performances," which is substantially broader than the concrete setlist-focused examples before it. Phrases like requests about "live performances" or "tours" could overlap with common music questions that do not actually require this skill, and no exclusion conditions or negative examples are provided.

Session Persistence

Medium
Category
Rogue Agent
Content
# setlist-mcp

MCP server for setlist.fm — search concert setlists, artists, venues, and tours via natural language. Read-only (setlist.fm has no write API).

- **npm:** [npmjs.com/package/setlist-mcp](https://www.npmjs.com/package/setlist-mcp)
- **Source:** [github.com/chrischall/setlist-mcp](https://github.com/chrischall/setlist-mcp)
Confidence
60% confidence
Finding
Skill establishes unauthorized persistence across sessions via cron jobs, startup scripts, or state files. Session persistence allows an attacker to maintain access beyond the current interaction.

Description-Behavior Mismatch

Medium
Confidence
97% confidence
Finding
The skill metadata and early documentation frame the server as read-only, but later it exposes attendance-marking tools that modify a user's setlist.fm account. This mismatch can mislead a caller, reviewer, or orchestration layer into invoking the skill under weaker safeguards than are appropriate for state-changing operations.

Intent-Code Divergence

Low
Confidence
95% confidence
Finding
The documentation says everything is read-only and then immediately lists two write tools, creating contradictory security expectations. Even if the write actions are confirm-gated, inconsistent documentation increases the chance of accidental use, improper trust decisions, or missing guardrails in systems that rely on the text description.

Static analysis

No suspicious patterns detected.