Back to skill

Security audit

Checkly

Security checks for vulnerabilities and agentic risk

Overview

The skill mostly fits a Checkly read-only helper, but its setup instructions tell the agent to execute a live remote installer without verification.

Review this skill before installing. Its normal Checkly read actions are coherent, but do not let an agent run the documented curl|bash or irm|iex installer automatically; install the oo CLI through a trusted, verified method and confirm any future action that could modify Checkly state.

Vulnerability Patterns
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
Findings (1)

T03 · Remote Payload Retrieval and Execution

Error
Location
SKILL.md:61
Finding
Unverified Remote Installation Scripts Executed Directly by Shells## Vulnerability Details **File Location**: `SKILL.md`, lines 61–65 **Vulnerability Type**: Remote payload retrieval and execution **Risk Level**: High The first-time setup instructions execute remotely hosted installation scripts directly in Bash and PowerShell: ```bash curl -fsSL https://cli.oomol.com/install.sh | bash # macOS / Linux ``` ```powershell irm https://cli.oomol.com/install.ps1 | iex # Windows PowerShell ``` ### Technical Analysis Both commands retrieve mutable content from an external server and pass it immediately to a command interpreter. The instructions provide no immutable version pinning, cryptographic signature verification, checksum validation, or opportunity to inspect the downloaded scripts before execution. Installation of the `oo` CLI supports the Skill's declared Checkly functionality. However, directly piping remote content into a shell exceeds the minimum privileges and trust required to install that dependency safely. The effective code executed can change at any time after this Skill has been reviewed. Successful exploitation would require control over, or compromise of, a relevant delivery point such as the remote hosting infrastructure, published installer, domain, DNS resolution, or trusted TLS path. A malicious or compromised installer could execute arbitrary commands with the privileges of the user running the setup command. ### Attack Path 1. The `oo` CLI is unavailable, and an agent or user follows the documented first-time setup procedure. 2. An attacker compromises or otherwise gains control over the installer delivered from `cli.oomol.com`, or a relevant part of its delivery chain. 3. The `curl` or `irm` command downloads attacker-controlled script content. 4. Bash or PowerShell executes the content immediately, without integrity validation or review. 5. The payload performs arbitrary actions using the invoking process's permissions, potentially including cred ...[truncated 872 chars]
Remediation
## Remediation Suggestions 1. Replace direct remote-to-shell execution with installation through a trusted package manager or a pinned, immutable release artifact. 2. Download the installer to a local file without executing it: ```bash curl -fLo install.sh https://cli.oomol.com/releases/<pinned-version>/install.sh ``` 3. Publish and verify a cryptographic signature from a separately distributed, trusted signing key. If signatures are unavailable, verify a pinned SHA-256 checksum obtained through an independent trusted channel. 4. Execute the installer only after successful verification and, where practical, manual inspection: ```bash sha256sum -c install.sh.sha256 bash install.sh ``` 5. Apply equivalent download, signature or checksum verification, and separate execution steps for the PowerShell installer. 6. Document the exact CLI version being installed and avoid mutable URLs such as a generic `install.sh` or `install.ps1`. 7. Run installation with the least-privileged account possible. Do not request administrator or root access unless a documented installation step strictly requires it. 8. Consider making installation a user-controlled prerequisite rather than allowing an automated agent to execute installer instructions.
Vulnerability Patterns
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
Findings (4)

Description-Behavior Mismatch

High
Confidence
97% confidence
Finding
The manifest promises a read/search-only Checkly skill, but the body instructs the agent to inspect and execute arbitrary connector actions, including possible state-changing ones. This scope mismatch can cause an orchestrator or user to trust the skill for low-risk read access while it actually enables broader operations, increasing the chance of unintended or unauthorized writes.

External Script Fetching

High
Category
Supply Chain
Content
- **`oo: command not found`** — install the oo CLI (other platforms: <https://cli.oomol.com/install-guide.md>):

  ```bash
  curl -fsSL https://cli.oomol.com/install.sh | bash    # macOS / Linux
  ```

  ```powershell
Confidence
96% confidence
Finding
The skill instructs users to install software by piping a remote script directly into the shell, which bypasses integrity verification and exposes users to supply-chain compromise if the server, transport, or script is tampered with. In a skill context, this is especially dangerous because it normalizes automated execution of fetched code as part of task completion.

Vague Triggers

Medium
Confidence
97% confidence
Finding
The instruction to use this skill for ANY Checkly request is overly broad and can override more specific, safer workflows or least-privilege routing. In context, that broad trigger matters more because the skill can reach arbitrary connector actions, so routine Checkly-related prompts may be funneled into a capability set broader than advertised.

Intent-Code Divergence

Medium
Confidence
84% confidence
Finding
The safety section claims write and destructive actions will be tagged, but the listed actions contain no such tags while the document also suggests arbitrary actions may exist via schema inspection. That inconsistency weakens operator safeguards because an agent may assume an untagged action is safe even when the tagging scheme is incomplete or inaccurate.

Static analysis

No suspicious patterns detected.