Back to skill

Security audit

Checkly

Security checks for vulnerabilities and agentic risk

Overview

The Checkly read operations fit the stated purpose, but automatic setup can execute unverified remote code and introduce broader agent changes without clear approval.

Review and approve the CLI installation separately before allowing automatic setup. Decide whether its agent-skill synchronization is acceptable and whether OOMOL may hold your Checkly credentials and process requests. Use a read-only Checkly API key. The bundled instructions show no malicious intent, but the remote installers and resulting executable were not verified.

Vulnerability Patterns
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
Vulnerability Patterns
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Output HandlingUnvalidated Output Injection, Cross-Context Output, Unbounded Output
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
Findings (4)

Description-Behavior Mismatch

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The supplied chunk runs an installer from a different service and contains no Checkly search or read operations. Nothing supplied establishes this installer as a necessary supporting dependency for Checkly functionality. The remote script's contents and downstream effects are not shown.

Content

No source excerpt is available for this finding.

External Script Fetching

High
Category
Supply Chain
Confidence
98% confidence
Finding

Piping a mutable remote script directly into bash executes its contents with the invoking user's permissions without prior verification or review. HTTPS protects transport but does not prevent a compromised distribution endpoint from serving malicious code; the legitimate dependency context and conditional trigger reduce exposure but do not remove that supply-chain risk.

Content

Scanner excerpt · SKILL.md (reported line 61)May include surrounding context.

The setup workflow executes downloaded installation scripts without warning about the system-integrity implications.

Medium
Category
Output Handling
Confidence
95% confidence
Finding

The fallback setup directs execution of downloaded shell or PowerShell code without explaining that this can modify the host or requesting installation approval. Although conditional and relevant to the workflow, installation crosses a substantially broader trust boundary than reading Checkly data and could expose local files or credentials if the installer is compromised.

Content

No source excerpt is available for this finding.

Autonomous Decision Making

Medium
Category
Excessive Agency
Confidence
80% confidence
Finding

Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.

Content

Scanner excerpt · SKILL.md (reported line 82)May include surrounding context.

md
- **`OOMOL_INSUFFICIENT_CREDIT`** — OOMOL billing stop. Recharge at `https://console.oomol.com/billing/token-recharge` before retrying.

- **HTTP 402 without `OOMOL_INSUFFICIENT_CREDIT`** — the third-party account itself has no active plan or credit. Report the provider error message as is; do not ask the user to recharge OOMOL.

## Resources