Checkly CLI Skills

• Outcome: Review
• Latest audit: Sep 9, 2026 · 3:54 PM UTC
• Version: 1.0.18

Overview

The skill mostly matches its Checkly CLI purpose, but it includes high-impact production/account operations and force-deploy guidance that can skip detailed review of destructive changes.

Review before installing. Use least-privilege Checkly keys, confirm the active account before writes, run deploy previews and review deletions before approving deployment, avoid the included force-deploy helper scripts unless your CI has a separate approval gate, pin Checkly package versions where possible, and protect any local config file or CI logs that may contain credentials.

A.I.G

Vulnerability Patterns

• Insecure Dependencies: Introduces malicious components through unsafe dependency sources
• Insecure Skill Coding Practices: Finds exploitable flaws such as hardcoded secrets or command injection
• Skill Instruction Hijacking: Alters the agent's session goals or safety constraints when the skill loads
• Agent Memory Poisoning: Writes attacker-controlled rules into memory that affect later sessions
• Remote Payload Retrieval and Execution: Fetches external code whose behavior can change after review

Findings (3)

T08 · Insecure Dependencies

Warning

• Location: scripts/init-project.sh:6
• Finding: Mutable npm Package Resolution Can Execute Unreviewed Remote Code<![CDATA[ ## Vulnerability Details **File Location**: `scripts/init-project.sh:6-13`; also referenced in `SKILL.md:19`, `SKILL.md:66`, and `SKILL.md:394` **Vulnerability Type**: Unpinned dependency execution **Risk Level**: Medium ### Vulnerable Code ```bash PROJECT_NAME="${1:-my-monitoring-project}" echo "🚀 Creating Checkly monitoring project: $PROJECT_NAME" # Create project npm create checkly@latest "$PROJECT_NAME" -- --yes cd "$PROJECT_NAME" ``` Related Skill instructions also recommend: ```bash npm create checkly@latest ``` ### Technical Analysis The `@latest` version selector resolves at execution time rather than identifying a version that was reviewed with this Skill. `npm create` downloads and executes package initialization code, so the effective code may change after the Skill itself has been audited. The repository also relies extensively on `npx checkly`. If a trusted project-local installation is unavailable, npm behavior and local configuration may permit package retrieval before execution. An unpinned invocation therefore expands trust from the reviewed Skill to the current npm registry state, package publisher account, dependency graph, and local npm configuration. This behavior is relevant to the Skill's declared Checkly setup functionality, but using a mutable package version is not the minimum privilege or trust necessary to perform that setup. ### Attack Path 1. An attacker compromises the relevant npm publisher, package release process, registry account, or a transitive dependency. 2. The attacker publishes a malicious version that becomes the package's `latest` release. 3. A user or agent invokes `scripts/init-project.sh` or follows the documented `npm create checkly@latest` instruction. 4. npm resolves and downloads the newly published package. 5. Package initialization code executes with the invoking user's local permissions. 6. Malicious code can access files and environment variables available to that process, potentially ...[truncated 737 chars]
• Remediation: <![CDATA[ ## Remediation Suggestions 1. Replace `@latest` with an exact, reviewed version: ```bash npm create checkly@9.3.0 "$PROJECT_NAME" -- --yes ``` 2. Record package versions and integrity data in a lockfile and use `npm ci` for established projects. 3. Prefer the project-local binary: ```bash ./node_modules/.bin/checkly test ``` or ensure `npx --no-install checkly` is used where supported. 4. Review release provenance, signatures, and npm integrity metadata before updating the pinned version. 5. Perform upgrades through a reviewed dependency-update process rather than resolving a mutable tag during routine Skill execution. 6. Run package installation with only the environment variables and filesystem access needed for initialization. ]]>

T09 · Insecure Skill Coding Practices

Error

• Location: scripts/test-and-deploy.sh:6
• Finding: Forced Deployment Workflow Bypasses Destructive Change Safeguards<![CDATA[ ## Vulnerability Details **File Location**: `scripts/test-and-deploy.sh:6-17`; related unsafe recommendation at `scripts/import-from-ui.sh:20-24` **Vulnerability Type**: Confirmation and destructive-deployment safeguard bypass **Risk Level**: High ### Vulnerable Code ```bash echo "🧪 Testing checks locally..." npx checkly test if [ $? -eq 0 ]; then echo "✅ All checks passed!" echo "" read -p "Deploy to production? (y/N) " -n 1 -r echo if [[ $REPLY =~ ^[Yy]$ ]]; then echo "🚀 Deploying checks..." npx checkly deploy --force echo "✅ Deployment complete!" ``` The import helper additionally recommends: ```bash echo "Next steps:" echo " 1. Review imported files: git diff" echo " 2. Test locally: npx checkly test" echo " 3. Commit to version control: git add . && git commit" echo " 4. Deploy to sync state: npx checkly deploy --force" ``` ### Technical Analysis The script obtains only a generic yes/no response and then invokes `npx checkly deploy --force`. According to the repository's own deployment documentation, `--force` skips confirmation prompts, including the destructive-delete guard. The workflow does not first execute and present an itemized `deploy --preview --verbose` result. Consequently, the user cannot determine from the prompt whether the deployment will create, update, detach, or permanently delete particular cloud resources. This conflicts with the agent-mode confirmation protocol in `SKILL.md`, which states that agents must present returned changes, obtain explicit approval, execute the returned confirmation command verbatim, and not append `--force` independently. Passing local tests does not establish that the deployment diff is safe. A valid project can still omit previously deployed resources, causing those resources and their run history to be deleted during a forced deployment. ### Attack Path 1. A malicious, compromised, or accidental project change removes one or more deployed Checkly resources f ...[truncated 1387 chars]
• Remediation: <![CDATA[ ## Remediation Suggestions 1. Remove the unconditional `--force` deployment from the interactive helper. 2. Generate an itemized preview before asking for approval: ```bash npx checkly deploy --preview --verbose ``` 3. Present all creates, updates, deletions, and detachments to the operator before requesting confirmation. 4. Run a normal non-forced deployment after approval, or use the exact `confirmCommand` returned by Checkly agent mode. 5. Do not construct, edit, or append `--force` to an agent-mode confirmation command. 6. Prefer resource preservation when definitions are intentionally removed: ```bash npx checkly deploy --preserve-resources ``` 7. For CI, require a reviewed deployment preview or plan artifact and protect production deployment with an approval gate. 8. Update `scripts/import-from-ui.sh` so its printed next steps recommend previewing and following the confirmation protocol rather than immediately using `--force`. ]]>

T09 · Insecure Skill Coding Practices

Warning

• Location: checkly-auth/SKILL.md:72
• Finding: Manual Authentication Guidance Stores API Keys in a Predictable Plaintext File Without Permission Hardening<![CDATA[ ## Vulnerability Details **File Location**: `checkly-auth/SKILL.md:72-78` **Vulnerability Type**: Plaintext credential storage without file-permission controls **Risk Level**: Medium ### Vulnerable Code ```markdown ### Configuration file (manual) Create/edit config file at `~/.config/@checkly/cli/config.json`: ```json { "apiKey": "cu_abc123...", "accountId": "12345" } ``` ``` ### Technical Analysis The manual authentication workflow instructs users to store a Checkly API key in a predictable plaintext JSON file. It does not require restrictive permissions, verify file ownership, use atomic creation, or warn that editor backups and filesystem backups may retain the secret. The example key is a placeholder rather than a hardcoded live credential. The vulnerability is therefore not credential inclusion in the repository; it is incomplete secret-storage guidance that may lead users or agents to create a sensitive file under a permissive umask. The document elsewhere recommends browser login and tells users not to commit credentials. Those safeguards reduce risk but do not address unauthorized local reads of the manually created configuration file. ### Attack Path 1. A user follows the manual configuration instructions. 2. The file is created under a permissive umask or by an editor that preserves broader permissions. 3. Another local account, compromised process, backup agent, or malware reads the predictable configuration path. 4. The attacker extracts the API key and account ID. 5. The attacker authenticates to Checkly as the associated user or service identity. 6. The attacker performs operations allowed by the key's assigned role. An alternative exposure path is accidental inclusion of the plaintext file or an editor backup in source control or support material. ### Impact Assessment The impact depends on the API key's role: - A read-only credential may expose monitoring configuration, results, account metadata, and failure artifacts ...[truncated 366 chars]
• Remediation: <![CDATA[ ## Remediation Suggestions 1. Retain browser-based `npx checkly login` as the preferred local-development method. 2. Prefer an operating-system credential store rather than plaintext JSON when the CLI supports it. 3. If file storage is unavoidable, document secure creation: ```bash install -d -m 700 "$HOME/.config/@checkly/cli" umask 077 ``` 4. Require the credential file to be owned by the current user and set to mode `0600`: ```bash chmod 600 "$HOME/.config/@checkly/cli/config.json" ``` 5. Warn users against placing the file inside a project directory or sharing it in logs, support archives, and backups. 6. Add the relevant configuration path and editor-backup patterns to exclusion and secret-scanning policies. 7. Use narrowly scoped service identities for automation and assign the least role required. 8. Rotate and revoke any key that may have been stored with unsafe permissions or committed to version control. ]]>

SkillSpector

Vulnerability Patterns

• Data Exfiltration: External Transmission, Env Variable Harvesting, File System Enumeration
• Tool Misuse: Tool Parameter Abuse, Chaining Abuse, Unsafe Defaults
• Trigger Abuse: Overly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
• MCP Least Privilege: Underdeclared Capability, Wildcard Permission, Missing Permission Declaration
• Prompt Injection: Instruction Override, Hidden Instructions, Exfiltration Commands

Findings (9)

Lp3

Medium

• Category: MCP Least Privilege
• Confidence: 86% confidence
• Finding: The skill declares required environment variables, including CHECKLY_API_KEY, but does not declare corresponding permissions despite relying on secret-bearing env access. This creates a capability/permission mismatch that can let an agent read sensitive credentials without an explicit least-privilege contract or user awareness.

Vague Triggers

Medium

• Confidence: 90% confidence
• Finding: The trigger list includes generic terms such as 'configuration,' 'project setup,' and 'defaults' that are not specific to Checkly. In an agent routing system, overly broad triggers can cause this skill to activate for unrelated requests, increasing the chance of incorrect guidance, context capture, or unintended tool behavior.

Vague Triggers

Medium

• Confidence: 90% confidence
• Finding: The trigger list includes broad phrases such as 'health check' and 'uptime monitoring' that can match many ordinary user requests outside the Checkly CLI domain. This can cause incorrect skill routing, making the agent invoke this skill in unrelated contexts and potentially provide irrelevant or unsafe guidance if the wrong operational domain is assumed.

Missing User Warnings

Medium

• Confidence: 88% confidence
• Finding: The skill explicitly states that verbose mode shows 'Environment variables used' during debugging, but it does not warn users that secrets from the shell or env file may be printed into local terminals, CI logs, or shared artifacts. In a CI/CD context, this can disclose API keys, tokens, or internal endpoints to anyone with log access.

External Transmission

Medium

• Category: Data Exfiltration

Content

setupScript: {
    content: `
      // Setup: Generate auth token
      const response = await fetch('https://api.example.com/auth/token', {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({

• Confidence: 94% confidence
• Finding: This duplicate finding captures the same risky setupScript behavior: outbound authentication using client credentials and runtime token propagation. The danger comes from embedding executable secret-exchange logic in a reusable skill example, which increases the chance of misuse or insecure adaptation.

External Transmission

Medium

• Category: Data Exfiltration

Content

new ApiCheck('authenticated-api-check', {
  name: 'Authenticated API',
  request: {
    url: 'https://api.example.com/user/profile',
    method: 'GET',
    headers: [
      { key: 'Authorization', value: 'Bearer {{API_TOKEN}}' },  // Your custom token

• Confidence: 89% confidence
• Finding: This example sends an Authorization header containing a bearer token to an external endpoint. Even though it is framed as user-defined configuration, the skill materially demonstrates external transmission of secrets and could lead users to paste production tokens into checks without sufficient warning about scope, storage, and destination trust.

External Transmission

Medium

• Category: Data Exfiltration

Content

setupScript: {
    content: `
      // Setup: Generate auth token
      const response = await fetch('https://api.example.com/auth/token', {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({

• Confidence: 94% confidence
• Finding: This duplicate finding captures the same risky setupScript behavior: outbound authentication using client credentials and runtime token propagation. The danger comes from embedding executable secret-exchange logic in a reusable skill example, which increases the chance of misuse or insecure adaptation.

External Transmission

Medium

• Category: Data Exfiltration

Content

`,
  },
  request: {
    url: 'https://api.example.com/protected',
    method: 'GET',
    headers: [
      { key: 'Authorization', value: 'Bearer {{AUTH_TOKEN}}' },

• Confidence: 87% confidence
• Finding: This request sends the previously acquired AUTH_TOKEN to an external protected endpoint. In context, it compounds the setupScript risk by showing a full credential acquisition and use flow, making it easier to operationalize secret-bearing outbound requests without guardrails.

Unsafe Defaults

Medium

• Category: Tool Misuse

Content

| `CHECKLY_ACCOUNT_ID` | Yes | Numeric account ID |
| `CHECKLY_API_URL` | No | Override the local API URL when `CHECKLY_ENV=local` (default: `http://127.0.0.1:3000`) |
| `CHECKLY_MQTT_URL` | No | Override the local events/MQTT broker when `CHECKLY_ENV=local` |
| `CHECKLY_SKIP_AUTH` | No | Skip authentication (for debugging flags) |

`CHECKLY_MQTT_URL` is advanced troubleshooting configuration for local or custom setups where test-session event streams come from a different broker. It is analogous to `CHECKLY_API_URL` for local endpoint overrides, not a normal hosted Checkly credential or default CI setting.

• Confidence: 96% confidence
• Finding: Documenting `CHECKLY_SKIP_AUTH` as an available flag normalizes an authentication-bypass mechanism without clearly constraining it to non-production or internal-only use. In an agent skill, this can lead users or downstream automation to disable auth during troubleshooting and accidentally run insecure workflows, especially if copied into CI/CD or shared scripts.

VirusTotal

VirusTotal engine telemetry is currently stale for this artifact.

Static analysis

No suspicious patterns detected.