Back to skill

Security audit

Langsmith Fetch

Security checks for vulnerabilities and agentic risk

Overview

This skill has a legitimate LangSmith debugging purpose, but it can automatically access sensitive trace data and includes unsafe credential-handling guidance.

Install only if you are comfortable giving the agent access to LangSmith traces for the selected project. Use explicit LangSmith-specific prompts, avoid printing API keys, do not store secrets in shell startup files unless you understand the exposure, review exported trace folders before sharing, and prefer pinned package versions or immutable release URLs.

Vulnerability Patterns
  • Insecure DependenciesIntroduces malicious components through unsafe dependency sources
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Findings (3)

T08 · Insecure Dependencies

Warning
Location
README.md:48
Finding

Unpinned Third-Party Package Installation

Content
View full analysis

Vulnerability Details

File Location: README.md:48-65; additional occurrences in README.md:261-265 and SKILL.md:22-24
Vulnerability Type: Supply-chain exposure through unpinned package installation
Risk Level: Medium

Vulnerable Code

bash
# Install langsmith-fetch CLI
pip install langsmith-fetch
bash
npx skills add OthmanAdi/langsmith-fetch-skill --skill langsmith-fetch -g

Additional upgrade instructions amplify the issue:

bash
# Reinstall if needed
pip install --upgrade langsmith-fetch

Technical Analysis

The installation commands retrieve mutable package versions without pinning a reviewed version or validating an integrity hash. Package-manager installation can execute package-controlled installation logic and subsequently places third-party executable code in the user's environment.

The global npx installation has an expanded scope because it modifies the user's global Agent Skill environment. The automatic upgrade command may replace a previously reviewed package with a newer, compromised release.

The dependency itself is necessary for the Skill's declared trace-fetching functionality. However, retrieving the latest available version is not the minimum privilege or trust necessary: the same functionality can be provided using a pinned, verified release.

Attack Path

  1. An attacker compromises the upstream package, its publishing credentials, a transitive dependency, or the package-resolution path.
  2. The attacker publishes a malicious version under the package name used by the documentation.
  3. A user follows pip install langsmith-fetch, pip install --upgrade langsmith-fetch, or the global npx installation instruction.
  4. The package manager resolves the mutable dependency to the malicious release.
  5. Malicious installation or runtime code executes with the permissions of the user running the package manager.
  6. The code can access file ...[truncated 568 chars]
Remediation
View remediation

Remediation Suggestions

  1. Pin langsmith-fetch to a specifically reviewed version rather than installing the latest release.
  2. Use Python hash verification, such as a locked requirements file containing --hash entries.
  3. Pin the Agent Skill installer and all relevant transitive dependencies.
  4. Avoid recommending unconditional --upgrade operations.
  5. Publish checksums or signatures for approved releases and document how users should verify them.
  6. Prefer an isolated virtual environment instead of global package installation.
  7. Document the expected package publisher, source repository, and release checksum so users can detect package substitution.

T08 · Insecure Dependencies

Warning
Location
README.md:73
Finding

Installation of a Mutable Remote Skill Definition

Content
View full analysis

Vulnerability Details

File Location: README.md:73-84
Vulnerability Type: Mutable remote dependency installed directly into an Agent Skill directory
Risk Level: Medium

Vulnerable Code

bash
#### Option A: Personal Skill (Just for you)
mkdir -p ~/.claude/skills/langsmith-fetch
curl -o ~/.claude/skills/langsmith-fetch/SKILL.md https://raw.githubusercontent.com/OthmanAdi/langsmith-fetch-skill/main/SKILL.md
bash
#### Option B: Project Skill (Share with team)
mkdir -p .claude/skills/langsmith-fetch
curl -o .claude/skills/langsmith-fetch/SKILL.md https://raw.githubusercontent.com/OthmanAdi/langsmith-fetch-skill/main/SKILL.md
git add .claude/skills/
git commit -m "Add LangSmith Fetch skill"
git push

Technical Analysis

These commands download SKILL.md from the mutable main branch and place it directly in a directory from which an Agent can load operational instructions. No commit pin, release tag, checksum, or signature verification is performed.

The downloaded file is not directly passed to a shell, and the reviewed version contains no malicious payload. Nevertheless, a future repository or branch compromise could modify the effective Skill after this audit. Once installed, altered instructions could cause the Agent to execute commands or disclose data under the authority granted to the Agent.

Downloading the Skill is necessary for installation, but trusting a mutable branch exceeds the minimum required trust. An immutable reviewed commit or signed release would provide the same functionality with a smaller supply-chain risk.

Attack Path

  1. An attacker compromises the upstream GitHub repository, maintainer credentials, or the main branch.
  2. The attacker replaces SKILL.md with instructions that misuse Agent tools, retrieve additional payloads, or expose sensitive information.
  3. A user executes the documented curl command.
  4. The malicious file is written dir ...[truncated 919 chars]
Remediation
View remediation

Remediation Suggestions

  1. Replace the mutable main URL with an immutable reviewed commit SHA or signed release artifact.
  2. Publish a SHA-256 checksum and require verification before moving the file into the active Skill directory.
  3. Download to a temporary staging location, inspect or validate the file, and only then install it.
  4. Use signed release tags and document signature verification.
  5. Do not automatically commit and push a remotely downloaded Skill before review.
  6. Clearly warn users that Agent Skill files are executable instructions and should be treated with the same caution as code.

T09 · Insecure Skill Coding Practices

Warning
Location
SKILL.md:33
Finding

LangSmith API Key Printed to Terminal Output

Content
View full analysis

Vulnerability Details

File Location: SKILL.md:33-36; repeated in SKILL.md:358-365 and README.md:238-241
Vulnerability Type: Plaintext credential disclosure
Risk Level: Medium

Vulnerable Code

bash
**Verify setup:**
```bash
echo $LANGSMITH_API_KEY
echo $LANGSMITH_PROJECT
text

The troubleshooting workflow repeats the same instruction:

```bash
# 2. Check environment
echo $LANGSMITH_API_KEY
echo $LANGSMITH_PROJECT

Technical Analysis

Printing LANGSMITH_API_KEY discloses the complete secret in terminal output. Verifying whether a variable is configured does not require revealing its value.

Terminal output can be captured by Agent tool logs, shell history mechanisms, continuous-integration logs, screen recordings, remote support sessions, shared terminals, or observability systems. Because this Skill is explicitly intended for an Agent-driven terminal environment, command output may also become part of the Agent's context or retained execution transcript.

The LangSmith API key is necessary to authenticate the declared trace-retrieval operations. Displaying the complete key is not necessary and exceeds minimum disclosure requirements.

Attack Path

  1. A user sets a valid LANGSMITH_API_KEY.
  2. The user or Agent follows the documented setup or troubleshooting instruction.
  3. echo $LANGSMITH_API_KEY writes the full credential to standard output.
  4. The output is captured in an Agent transcript, terminal log, CI log, screen share, screenshot, or other retained record.
  5. An unauthorized party gains access to that record.
  6. The party uses the disclosed key against LangSmith APIs within the key's authorized scope.

Impact Assessment

Exposure may allow unauthorized access to LangSmith projects and trace data available to the compromised key. Depending on the key's permissions, an attacker could retrieve sensitive prompts, model outputs, tool inputs and resu ...[truncated 265 chars]

Remediation
View remediation

Remediation Suggestions

  1. Remove every instruction that prints the complete API key.
  2. Verify only whether the variable is set:
    bash
    if [ -n "${LANGSMITH_API_KEY:-}" ]; then
      echo "LANGSMITH_API_KEY is set"
    else
      echo "LANGSMITH_API_KEY is not set"
    fi
    
  3. If identifying a key is essential, reveal only a short suffix and mask the remaining characters.
  4. Advise users to rotate any key that has already appeared in logs, screenshots, transcripts, or shared terminal output.
  5. Prevent secret-bearing command output from being included in Agent context or persistent diagnostic exports.
  6. Recommend least-privilege LangSmith keys with only the permissions required to retrieve the intended traces.
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Rogue AgentSelf-Modification, Session Persistence
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
Findings (25)

Credential Access

High
Category
Privilege Escalation
Confidence
60% confidence
Finding

Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Content

Scanner excerpt · .gitignore (reported line 42)May include surrounding context.

text
yarn.lock

# Environment variables (security)
.env
.env.local
.env.*.local

Credential Access

High
Category
Privilege Escalation
Confidence
60% confidence
Finding

Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Content

Scanner excerpt · .gitignore (reported line 43)May include surrounding context.

text
# Environment variables (security)
.env
.env.local
.env.*.local

YARA rule 'agent_skill_mcp_tool_poisoning_metadata': MCP/tool metadata poisoning indicators in tool schemas or skill manifests [agent_skills]

High
Category
YARA Match
Confidence
80% confidence
Finding

YARA rule matched a hack tool or exploit indicator (offensive tools, reconnaissance, privilege escalation, or exploit frameworks).

Content

Scanner excerpt · README.md (reported line 112)May include surrounding context.

git commit -m "Add LangSmith Fetch skill" git push

text

</details>

### 4. Use with Claude

Open Claude Code and ask:

Debug my agent - what happened in the last 5 minutes?

text

Claude will automatically use this skill! 🎉

---

## 💡 Usage Examples

### Example 1: Quick Debug

You: "Debug my agent"

Claude: Found 3 traces in the last 5 minutes:

Trace 1: ✅ Success

  • Agent: memento
  • Tools: recall_memories, create_entities
  • Duration: 2.3s
  • Tokens: 1,245

Trace 2: ❌ Error

  • Agent: cypher
  • Error: "Neo4j connection timeout"
  • Duration: 15.1s
  • Failed at: search_nodes tool

💡 Issue: Neo4j timeout. Check database connection and add retry logic.

text

### Example 2: Error Investigation

You: "Why did trace abc123 fail?"

Claude: Deep Dive Analysis - Trace abc123

Goal: User asked "Find all projects"

Execution Flow:

  1. ✅ search_nodes → Found 24 nodes
  2. ❌ get_node_details → Error: "Node not found"

Root Cause: Search returned deleted node IDs

Suggested Fix: 1.

YARA rule 'backdoor_persistence': Backdoor persistence with malicious payloads (shell commands, SSH key injection, hidden root users) [malware]

High
Category
YARA Match
Confidence
75% confidence
Finding

YARA rule matched a known malware signature (reverse shell, backdoor, ransomware, C2 framework, or info stealer).

Content

Scanner excerpt · README.md (reported line 224)May include surrounding context.

config # Manage configuration

text

---

## 🛠️ Configuration

### Environment Variables

**Required:**
```bash
LANGSMITH_API_KEY    # Your LangSmith API key
LANGSMITH_PROJECT    # Your project name

Optional:

bash
LANGCHAIN_ENDPOINT   # Custom endpoint (default: https://api.smith.langchain.com)

Making Variables Persistent

Add to ~/.bashrc or ~/.zshrc:

bash
echo 'export LANGSMITH_API_KEY="your_key"' >> ~/.bashrc
echo 'export LANGSMITH_PROJECT="your_project"' >> ~/.bashrc
source ~/.bashrc

🔍 Troubleshooting

"No traces found"

Cause: No recent agent activity or tracing disabled

Fix:

bash
# Check environment
echo $LANGSMITH_API_KEY
echo $LANGSMITH_PROJECT

# Try longer timeframe
langsmith-fetch traces --last-n-minutes 1440 --limit 50

# Verify tracing is enabled
# In your code: LANGCHAIN_TRACING_V2=true

Skill not activating

Fix:

  1. Ensure SKILL.md is in ~/.claude/skills/langsmith-fetch/
  2. Restart Cl

YARA rule 'backdoor_persistence': Backdoor persistence with malicious payloads (shell commands, SSH key injection, hidden root users) [malware]

High
Category
YARA Match
Confidence
75% confidence
Finding

YARA rule matched a known malware signature (reverse shell, backdoor, ransomware, C2 framework, or info stealer).

Content

Scanner excerpt · SKILL.md (reported line 393)May include surrounding context.

config # Manage configuration

text

---

## 🛠️ Configuration

### Environment Variables

**Required:**
```bash
LANGSMITH_API_KEY    # Your LangSmith API key
LANGSMITH_PROJECT    # Your project name

Optional:

bash
LANGCHAIN_ENDPOINT   # Custom endpoint (default: https://api.smith.langchain.com)

Making Variables Persistent

Add to ~/.bashrc or ~/.zshrc:

bash
echo 'export LANGSMITH_API_KEY="your_key"' >> ~/.bashrc
echo 'export LANGSMITH_PROJECT="your_project"' >> ~/.bashrc
source ~/.bashrc

🔍 Troubleshooting

"No traces found"

Cause: No recent agent activity or tracing disabled

Fix:

bash
# Check environment
echo $LANGSMITH_API_KEY
echo $LANGSMITH_PROJECT

# Try longer timeframe
langsmith-fetch traces --last-n-minutes 1440 --limit 50

# Verify tracing is enabled
# In your code: LANGCHAIN_TRACING_V2=true

Skill not activating

Fix:

  1. Ensure SKILL.md is in ~/.claude/skills/langsmith-fetch/
  2. Restart Cl

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
88% confidence
Finding

The changelog advertises exporting debug sessions to organized folders but provides no warning that this writes potentially sensitive trace contents to local disk. Trace exports may include prompts, tool inputs/outputs, memory contents, identifiers, and other sensitive operational data, so silent or poorly signposted file creation can lead to inadvertent persistence and broader exposure.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
96% confidence
Finding

The listed activation phrases are broad, generic troubleshooting requests such as 'What went wrong?' and 'Check for errors', which can cause the skill to trigger in contexts the user did not specifically intend. In a skill that fetches execution traces from LangSmith, accidental activation can expose debugging data, tool-call history, memory operations, and project metadata more often than necessary, increasing the risk of unnecessary data access.

Content

No source excerpt is available for this finding.

Session Persistence

Medium
Category
Rogue Agent
Confidence
60% confidence
Finding

Skill establishes unauthorized persistence across sessions via cron jobs, startup scripts, or state files. Session persistence allows an attacker to maintain access beyond the current interaction.

Content

Scanner excerpt · CONTRIBUTING.md (reported line 30)May include surrounding context.

cd langsmith-fetch-skill

text

### 3. Create a Branch

```bash
git checkout -b feature/your-feature-name

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

The README says Claude will 'automatically fetch and analyze execution traces' but does not define boundaries for when activation should or should not occur. In a debugging/observability skill, ambiguity is risky because traces may contain prompts, tool inputs, outputs, metadata, and other sensitive operational details.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
97% confidence
Finding

The README advertises automatic fetching of execution traces and session export without a clear warning that traces and saved sessions may contain sensitive prompts, tool results, identifiers, or secrets. Users may not realize that enabling this workflow can expose or persist operationally sensitive data.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The trigger phrases are broad everyday debugging requests such as 'Debug my agent' and 'What went wrong?', which can cause the skill to activate in contexts where the user did not explicitly consent to fetching traces. Because the skill accesses potentially sensitive execution traces, over-broad activation increases the risk of unintended data access and disclosure.

Content

No source excerpt is available for this finding.

Rp1

Medium
Category
MCP Rug Pull
Confidence
70% confidence
Finding

npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.

Content

No source excerpt is available for this finding.

Skill Enumeration

Medium
Category
Agent Snooping
Confidence
85% confidence
Finding

Skill enumerates or reads other installed skills. Access to other skills' SKILL.md files or the skills directory reveals prompt instructions, capabilities, and secrets that should be invisible to peer skills.

Content

Scanner excerpt · CONTRIBUTING.md (reported line 78)May include surrounding context.

Option A: Personal Skill (Just for you)

bash
mkdir -p ~/.claude/skills/langsmith-fetch
curl -o ~/.claude/skills/langsmith-fetch/SKILL.md https://raw.githubusercontent.com/OthmanAdi/langsmith-fetch-skill/main/SKILL.md

Skill Enumeration

Medium
Category
Agent Snooping
Confidence
85% confidence
Finding

Skill enumerates or reads other installed skills. Access to other skills' SKILL.md files or the skills directory reveals prompt instructions, capabilities, and secrets that should be invisible to peer skills.

Content

Scanner excerpt · README.md (reported line 75)May include surrounding context.

Option A: Personal Skill (Just for you)

bash
mkdir -p ~/.claude/skills/langsmith-fetch
curl -o ~/.claude/skills/langsmith-fetch/SKILL.md https://raw.githubusercontent.com/OthmanAdi/langsmith-fetch-skill/main/SKILL.md

Session Persistence

Medium
Category
Rogue Agent
Confidence
60% confidence
Finding

Skill establishes unauthorized persistence across sessions via cron jobs, startup scripts, or state files. Session persistence allows an attacker to maintain access beyond the current interaction.

Content

Scanner excerpt · README.md (reported line 75)May include surrounding context.

Option A: Personal Skill (Just for you)

bash
mkdir -p ~/.claude/skills/langsmith-fetch
curl -o ~/.claude/skills/langsmith-fetch/SKILL.md https://raw.githubusercontent.com/OthmanAdi/langsmith-fetch-skill/main/SKILL.md

Skill Enumeration

Medium
Category
Agent Snooping
Confidence
80% confidence
Finding

Skill enumerates or reads other installed skills. Access to other skills' SKILL.md files or the skills directory reveals prompt instructions, capabilities, and secrets that should be invisible to peer skills.

Content

Scanner excerpt · README.md (reported line 76)May include surrounding context.

Option A: Personal Skill (Just for you)

bash
mkdir -p ~/.claude/skills/langsmith-fetch
curl -o ~/.claude/skills/langsmith-fetch/SKILL.md https://raw.githubusercontent.com/OthmanAdi/langsmith-fetch-skill/main/SKILL.md

Option B: Project Skill (Share with team)

Skill Enumeration

Medium
Category
Agent Snooping
Confidence
80% confidence
Finding

Skill enumerates or reads other installed skills. Access to other skills' SKILL.md files or the skills directory reveals prompt instructions, capabilities, and secrets that should be invisible to peer skills.

Content

Scanner excerpt · README.md (reported line 82)May include surrounding context.

Option A: Personal Skill (Just for you)

bash
mkdir -p ~/.claude/skills/langsmith-fetch
curl -o ~/.claude/skills/langsmith-fetch/SKILL.md https://raw.githubusercontent.com/OthmanAdi/langsmith-fetch-skill/main/SKILL.md

Option B: Project Skill (Share with team)

Skill Enumeration

Medium
Category
Agent Snooping
Confidence
80% confidence
Finding

Skill enumerates or reads other installed skills. Access to other skills' SKILL.md files or the skills directory reveals prompt instructions, capabilities, and secrets that should be invisible to peer skills.

Content

Scanner excerpt · README.md (reported line 76)May include surrounding context.

Option A: Personal Skill (Just for you)

bash
mkdir -p ~/.claude/skills/langsmith-fetch
curl -o ~/.claude/skills/langsmith-fetch/SKILL.md https://raw.githubusercontent.com/OthmanAdi/langsmith-fetch-skill/main/SKILL.md

Option B: Project Skill (Share with team)

Skill Enumeration

Medium
Category
Agent Snooping
Confidence
80% confidence
Finding

Skill enumerates or reads other installed skills. Access to other skills' SKILL.md files or the skills directory reveals prompt instructions, capabilities, and secrets that should be invisible to peer skills.

Content

Scanner excerpt · README.md (reported line 82)May include surrounding context.

Option A: Personal Skill (Just for you)

bash
mkdir -p ~/.claude/skills/langsmith-fetch
curl -o ~/.claude/skills/langsmith-fetch/SKILL.md https://raw.githubusercontent.com/OthmanAdi/langsmith-fetch-skill/main/SKILL.md

Option B: Project Skill (Share with team)

Skill Enumeration

Medium
Category
Agent Snooping
Confidence
85% confidence
Finding

Skill enumerates or reads other installed skills. Access to other skills' SKILL.md files or the skills directory reveals prompt instructions, capabilities, and secrets that should be invisible to peer skills.

Content

Scanner excerpt · README.md (reported line 81)May include surrounding context.

Option B: Project Skill (Share with team)

bash
mkdir -p .claude/skills/langsmith-fetch
curl -o .claude/skills/langsmith-fetch/SKILL.md https://raw.githubusercontent.com/OthmanAdi/langsmith-fetch-skill/main/SKILL.md
git add .claude/skills/
git commit -m "Add LangSmith Fetch skill"

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
96% confidence
Finding

The 'How It Works' section describes Claude 'listens' for debugging-related questions and 'automatically activates' the skill, but gives no negative examples or guardrails. This creates a realistic risk of unintended invocation and unnecessary retrieval of sensitive trace data during normal conversation about bugs or performance.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
92% confidence
Finding

The skill description does not warn that traces and threads may contain sensitive execution data, prompts, tool inputs, memory contents, metadata, or tokens/cost details. Because the skill is designed to fetch and export debugging artifacts, lack of disclosure increases the chance of over-collection or inappropriate sharing of sensitive information.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
96% confidence
Finding

The activation phrases are very broad, such as 'What happened?' and 'What went wrong?', which can match ordinary conversation and trigger the skill unintentionally. In this skill's context, accidental activation can cause sensitive trace and thread data to be fetched or exported without a clear user request for that access.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

The skill includes commands to permanently alter LangSmith CLI configuration, which exceeds the narrow purpose of fetching and analyzing traces. Persistent configuration changes can unexpectedly affect other projects, workflows, or future agent actions, and are risky when embedded in an auto-invoked skill.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The skill instructs the agent to append environment variables, including an API key, to persistent shell startup files and reload them. This goes beyond transient debugging and creates long-lived credential exposure and host modification risk, especially if an agent follows the instructions automatically on a user's machine.

Content

No source excerpt is available for this finding.