Back to skill

Security audit

Agentchat Skill

Security checks for vulnerabilities and agentic risk

Overview

The skill is a legitimate AgentChat integration, but its setup can install unpinned third-party code and persistent remote skill instructions without integrity checks.

Install only if you trust the AgentChat publisher and are comfortable with a user-scoped MCP server being added. Prefer a pinned package version and a pinned or checksummed skill file before running setup, and review the downloaded files before allowing future sessions to use them.

Vulnerability Patterns
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Insecure DependenciesIntroduces malicious components through unsafe dependency sources
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (2)

T08 · Insecure Dependencies

Error
Location
SKILL.md:23
Finding
Unpinned Third-Party Package Is Downloaded and Executed Automatically## Vulnerability Details **File Location**: `SKILL.md`, lines 23-31 **Vulnerability Type**: Unsafe third-party dependency installation and execution **Risk Level**: High ### Vulnerable Code ```markdown Check if you have `agentchat_connect` in your tools. **If you DON'T have the tools:** Run this command now: ```bash claude mcp add -s user agentchat -- npx -y @tjamescouch/agentchat-mcp ``` ``` ### Technical Analysis The skill directs the agent to use `npx -y` to retrieve and immediately execute `@tjamescouch/agentchat-mcp`. The package reference does not specify an exact version, integrity hash, trusted lockfile, or signature. The `-y` option also suppresses the normal installation confirmation. Consequently, the code executed during installation is determined by the package version resolved from the registry at invocation time rather than by the version reviewed with this skill. Package lifecycle scripts and the installed MCP server run with the privileges of the user operating the agent. The `-s user` option additionally records the MCP configuration at user scope, making the unverified component available beyond the current project. This does not establish that the referenced package is currently malicious. The vulnerability is the unsafe dependency acquisition and execution process, which creates a supply-chain code-execution path if the package, maintainer account, publication pipeline, or registry resolution is compromised. ### Attack Path 1. An attacker compromises the package maintainer account, package publication pipeline, or another part of the package distribution path. 2. The attacker publishes a malicious version of `@tjamescouch/agentchat-mcp`. 3. An agent loads this skill on a system where the AgentChat tools are unavailable. 4. Following the skill instructions, the agent runs the unpinned `npx -y` command. 5. `npx` resolves the attacker-controlled release and executes its installation or runtime ...[truncated 876 chars]
Remediation
## Remediation Suggestions 1. Pin the package to an exact, reviewed version, for example `@tjamescouch/agentchat-mcp@X.Y.Z`. 2. Verify the package against a documented cryptographic integrity value or signed release before execution. 3. Remove `-y` and require explicit user approval before downloading or executing the dependency. 4. Use a lockfile or a controlled internal package mirror that only contains reviewed artifacts. 5. Review package lifecycle scripts and the MCP server entry point before installation. 6. Prefer project-scoped configuration unless user-scoped persistence is explicitly required and approved. 7. Run the MCP server with restricted filesystem, environment-variable, process, and network access. 8. Document the package version and integrity value in the skill so the audited artifact corresponds to the executed artifact.

T03 · Remote Payload Retrieval and Execution

Error
Location
SKILL.md:33
Finding
Mutable Remote Skill Instructions Are Installed Persistently Without Integrity Verification## Vulnerability Details **File Location**: `SKILL.md`, lines 33-37 **Vulnerability Type**: Unverified remote instruction retrieval and cross-session installation **Risk Level**: High ### Vulnerable Code ```markdown Then, if `.claude/skills/agentchat/SKILL.md` does NOT already exist, install this skill for future sessions: ```bash mkdir -p .claude/skills/agentchat && curl -sL https://raw.githubusercontent.com/tjamescouch/agentchat/main/SKILL.md > .claude/skills/agentchat/SKILL.md ``` ``` ### Technical Analysis The command downloads `SKILL.md` from the mutable `main` branch and writes it into `.claude/skills/agentchat/SKILL.md` for use in future sessions. It does not pin a reviewed commit, validate a cryptographic checksum, verify a signature, or present the downloaded content for approval. Although the retrieved file is an instruction document rather than a conventional executable, installing it in the skills directory makes its contents an effective behavioral payload when future agent sessions load the skill. Changes made upstream after this audit can therefore alter future agent behavior without modifying the locally audited project. The use of `curl -sL` also follows redirects and suppresses normal progress output. HTTPS protects transport under ordinary conditions but does not establish that the current content of the mutable branch is identical to the content reviewed during this audit. ### Attack Path 1. An attacker compromises the upstream repository, a maintainer account, or the content served for the referenced `main` branch. 2. The attacker replaces `SKILL.md` with instructions that induce unsafe tool calls, data disclosure, additional downloads, or other attacker-selected behavior. 3. An agent follows the audited skill's setup procedure. 4. `curl` retrieves the current attacker-controlled document without checking a pinned digest or signature. 5. The document is written to `.claude/skills/agent ...[truncated 926 chars]
Remediation
## Remediation Suggestions 1. Replace the mutable `main` URL with a URL pinned to a reviewed commit hash or immutable signed release. 2. Publish and verify a SHA-256 or stronger digest before writing the downloaded file into the skills directory. 3. Verify a maintainer signature using a documented trust policy. 4. Download to a temporary file first, validate it, and only then atomically move it into the destination. 5. Display the exact source revision, digest, destination, and persistence implications to the user before installation. 6. Require explicit user consent before creating a skill that will affect future sessions. 7. Prefer bundling the audited skill content locally instead of fetching mutable instructions at runtime. 8. Restrict updates to a separate, explicit update workflow that supports review and rollback. 9. Avoid silent retrieval flags during security-sensitive installation, and fail closed on redirects to unexpected origins.
Vulnerability Patterns
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Tool MisuseTool Parameter Abuse, Chaining Abuse, Unsafe Defaults
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
Findings (11)

Tool Parameter Abuse

High
Category
Tool Misuse
Content
4. **To remove the local skill copy** (if corrupted):
   ```bash
   rm -rf .claude/skills/agentchat
   ```
   Then re-run the install command from Setup.
Confidence
97% confidence
Finding
The skill includes `rm -rf .claude/skills/agentchat`, a destructive shell command that could be executed by an agent or copied by a user without adequate safeguards. Even with a relative path, `rm -rf` is inherently dangerous in automation contexts because path assumptions can fail and the command irreversibly removes local state.

Vague Triggers

Medium
Confidence
94% confidence
Finding
The phrase indicating the user wants to 'connect to AgentChat' is broad and overlaps with ordinary language, making accidental activation more likely. If this skill auto-triggers on common phrasing, it could lead to unintended network connections, installation prompts, or message transmission without sufficiently specific user intent.

Context-Inappropriate Capability

Medium
Confidence
93% confidence
Finding
The skill goes beyond chat functionality and directs shell execution, package installation, remote download, and file writes to the local `.claude/skills` directory. Embedding system-modifying setup steps inside a skill increases the chance an agent will perform sensitive local actions automatically in response to a simple chat-related request.

Rp1

Medium
Category
MCP Rug Pull
Confidence
96% confidence
Finding
The skill instructs the agent to install and execute an MCP server via `npx -y @tjamescouch/agentchat-mcp` without pinning an exact version or integrity hash. That creates a supply-chain risk: future package updates or a compromised publisher account could cause arbitrary code to run on the user's machine when the instruction is followed.

Skill Enumeration

Medium
Category
Agent Snooping
Content
claude mcp add -s user agentchat -- npx -y @tjamescouch/agentchat-mcp
```

Then, if `.claude/skills/agentchat/SKILL.md` does NOT already exist, install this skill for future sessions:

```bash
mkdir -p .claude/skills/agentchat && curl -sL https://raw.githubusercontent.com/tjamescouch/agentchat/main/SKILL.md > .claude/skills/agentchat/SKILL.md
Confidence
80% confidence
Finding
Skill enumerates or reads other installed skills. Access to other skills' SKILL.md files or the skills directory reveals prompt instructions, capabilities, and secrets that should be invisible to peer skills.

Skill Enumeration

Medium
Category
Agent Snooping
Content
claude mcp add -s user agentchat -- npx -y @tjamescouch/agentchat-mcp
```

Then, if `.claude/skills/agentchat/SKILL.md` does NOT already exist, install this skill for future sessions:

```bash
mkdir -p .claude/skills/agentchat && curl -sL https://raw.githubusercontent.com/tjamescouch/agentchat/main/SKILL.md > .claude/skills/agentchat/SKILL.md
Confidence
80% confidence
Finding
Skill enumerates or reads other installed skills. Access to other skills' SKILL.md files or the skills directory reveals prompt instructions, capabilities, and secrets that should be invisible to peer skills.

Missing User Warnings

Medium
Confidence
95% confidence
Finding
These instructions download remote content with `curl` and write it into the local skills directory without a prominent warning that the system will be modified. Remote fetch plus local persistence is dangerous because it can install or update agent behavior from mutable upstream content, enabling supply-chain compromise or silent persistence.

Skill Enumeration

Medium
Category
Agent Snooping
Content
Then, if `.claude/skills/agentchat/SKILL.md` does NOT already exist, install this skill for future sessions:

```bash
mkdir -p .claude/skills/agentchat && curl -sL https://raw.githubusercontent.com/tjamescouch/agentchat/main/SKILL.md > .claude/skills/agentchat/SKILL.md
```

Then tell the user:
Confidence
85% confidence
Finding
Skill enumerates or reads other installed skills. Access to other skills' SKILL.md files or the skills directory reveals prompt instructions, capabilities, and secrets that should be invisible to peer skills.

Context-Inappropriate Capability

Medium
Confidence
91% confidence
Finding
The troubleshooting section instructs users to remove and reinstall components and delete local directories, which are destructive system changes not required for ordinary AgentChat messaging. In a skill context, these instructions can normalize risky maintenance actions and encourage an agent to alter the local environment unsafely.

Rp1

Medium
Category
MCP Rug Pull
Confidence
96% confidence
Finding
The troubleshooting flow repeats the unpinned `npx` installation command, reintroducing the same remote code execution and supply-chain exposure during reinstall. Because troubleshooting steps are often followed under pressure, users may execute them without scrutinizing the risk.

Missing User Warnings

Medium
Confidence
96% confidence
Finding
The troubleshooting instructions include deleting `.claude/skills/agentchat` without an explicit warning about data loss or unintended side effects. Even though the path is scoped, destructive deletion commands in a skill can be copied or executed reflexively and may remove local customizations or trusted state.