Back to skill

Security audit

LangSmith

Security checks for vulnerabilities and agentic risk

Overview

LangSmith access is clearly explained, but setup can execute unverified internet code without asking you first.

Review and verify the CLI installer before authorizing setup, including any additional agent skills it installs or updates. Use a LangSmith API key limited to the intended work and understand that requests pass through OOMOL. The remote installation and authentication implementations are not bundled, so their exact system changes and credential storage remain unverified.

Vulnerability Patterns
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
Vulnerability Patterns
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Output HandlingUnvalidated Output Injection, Cross-Context Output, Unbounded Output
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
Findings (5)

Description-Behavior Mismatch

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The supplied command accesses Oomol infrastructure and executes its installer. The description does not declare Oomol installation, and the supplied code establishes no connection between this installation and LangSmith functionality. The downloaded script is not provided, so its further behavior cannot be assessed.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

High
Category
MCP Tool Poisoning
Confidence
82% confidence
Finding

The supplied code performs only a CLI login, with no visible connection to LangSmith or its data operations. Authentication could be a supporting implementation detail if oo were established as the LangSmith access mechanism, but the supplied material does not establish that relationship.

Content

No source excerpt is available for this finding.

External Script Fetching

High
Category
Supply Chain
Confidence
98% confidence
Finding

The command pipes a mutable remote script directly into bash without pinning or independently verifying its contents; the PowerShell alternative has the same problem. If the distribution endpoint or release pipeline is compromised, an agent following this fallback could execute attacker-controlled code with its local user's privileges, potentially exposing files and credentials.

Content

Scanner excerpt · SKILL.md (reported line 64)May include surrounding context.

The setup workflow executes downloaded installation scripts without warning the user about remote code execution or system changes.

Medium
Category
Output Handling
Confidence
96% confidence
Finding

The missing-CLI fallback instructs the agent to execute a remote installer without first explaining system changes or obtaining installation-specific approval. Although conditional and relevant to setup, it expands a LangSmith data request into local code execution and software installation without a comparable consent safeguard to the one required for LangSmith writes.

Content

No source excerpt is available for this finding.

Autonomous Decision Making

Medium
Category
Excessive Agency
Confidence
80% confidence
Finding

Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.

Content

Scanner excerpt · SKILL.md (reported line 85)May include surrounding context.

md
- **`OOMOL_INSUFFICIENT_CREDIT`** — OOMOL billing stop. Recharge at `https://console.oomol.com/billing/token-recharge` before retrying.

- **HTTP 402 without `OOMOL_INSUFFICIENT_CREDIT`** — the third-party account itself has no active plan or credit. Report the provider error message as is; do not ask the user to recharge OOMOL.

## Resources