Security audit

Pilot Alert

Security checks for vulnerabilities and agentic risk

Overview

This alerting skill is coherent and purpose-aligned, but users should sanitize alert contents and harden the example shell snippets before using them in production.

Install only if you intend to let the agent subscribe to Pilot events and send alerts to configured recipients. Before production use, restrict subscribed topics and webhook destinations, redact or minimize payload fields, avoid sending secrets to third-party webhooks, build JSON with jq, and replace the /tmp deduplication cache with a private locked state directory.

A.I.G

Vulnerability Patterns

Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally

Findings (3)

T09 · Insecure Skill Coding Practices

Warning

Location: SKILL.md:73
Finding: Unsafe JSON Construction from Untrusted Event Data
Content: View full analysis
Remediation: View remediation
/dev/null 2>&1 <<<"$pilot_payload"; then echo "Refusing to send invalid alert payload" >&2 continue fi ``` 5. Ensure the receiving agent validates the expected schema and does not treat alert text as executable instructions. ]]>

T09 · Insecure Skill Coding Practices

Warning

Location: SKILL.md:113
Finding: Predictable Shared Temporary Files Permit Symlink and Race Attacks
Content: View full analysis
/dev/null | cut -d' ' -f2) if [ -n "$last_sent" ] && [ $((now - last_sent)) -lt $DEDUP_WINDOW ]; then echo "Suppressed duplicate" continue fi # Send alert pilotctl --json send-message oncall-agent --data "$(echo "$event" | jq -r '.data')" # Update cache grep -v "^$alert_key " "$ALERT_FILE" > "${ALERT_FILE}.tmp" 2>/dev/null || true echo "$alert_key $now" >> "${ALERT_FILE}.tmp" mv "${ALERT_FILE}.tmp" "$ALERT_FILE" done ``` ### Technical Analysis The workflow uses globally predictable paths in the shared `/tmp` directory: - `/tmp/alert-cache.txt` - `/tmp/alert-cache.txt.tmp` It neither creates a private directory nor verifies that these paths are regular files owned by the current user. Shell output redirection follows symbolic links, so a local attacker may pre-create the `.tmp` path as a symlink to another writable file. The `grep` redirection can then truncate that target, and the subsequent append can modify it using the privileges of the process running the workflow. The read-modify-move sequence is also not protected by a lock. Multiple workflow instances can read stale state, overwrite each other's updates, or produce inconsistent deduplication decisions. ### Attack Path A local symlink attack can proceed as follows: 1. The attacker predicts the fixed `/tmp/alert-cache.txt.tmp` path. 2. The attacker creates that path as a symbolic ...[truncated 1564 chars]
Remediation: View remediation
"$tmp_file" 2>/dev/null || true printf '%s %s\n' "$alert_key" "$now" >> "$tmp_file" mv -- "$tmp_file" "$ALERT_FILE" ) 9>"$LOCK_FILE" ``` ]]>

T09 · Insecure Skill Coding Practices

Note

Location: SKILL.md:119
Finding: Event-Controlled Alert Key Is Interpreted as a Regular Expression
Content: View full analysis
/dev/null | cut -d' ' -f2) if [ -n "$last_sent" ] && [ $((now - last_sent)) -lt $DEDUP_WINDOW ]; then echo "Suppressed duplicate" continue fi # Send alert pilotctl --json send-message oncall-agent --data "$(echo "$event" | jq -r '.data')" # Update cache grep -v "^$alert_key " "$ALERT_FILE" > "${ALERT_FILE}.tmp" 2>/dev/null || true echo "$alert_key $now" >> "${ALERT_FILE}.tmp" ``` ### Technical Analysis The `alert_key` begins with the event's `.topic` value. That value is inserted directly into a `grep` basic regular expression: ```bash grep "^$alert_key " grep -v "^$alert_key " ``` Regular-expression metacharacters in the topic—such as `.`, `*`, `[`, `]`, `^`, or `$`—are interpreted as pattern syntax rather than literal text. Consequently, an attacker-controlled topic can match cache entries other than its own. The anchored prefix does not prevent this issue because the attacker controls pattern syntax immediately after the anchor. The same injected pattern is used both when checking the prior send time and when deleting existing entries during cache replacement. ### Attack Path 1. An attacker who can publish events chooses a topic containing regular-expression metacharacters. 2. The topic is incorporated into `alert_key` without escaping or encoding. 3. `grep "^$alert_key "` interprets the crafted topic as a regular expression. 4. The pattern matches an unrelated cache record. 5. The workflow may use that record's timestamp and incorrectly suppress the attacker's event. 6. During the update, `grep -v "^$alert_key "` may remove ...[truncated 757 chars]
Remediation: View remediation
"$tmp_file" 2>/dev/null || true printf '%s %s\n' "$alert_key" "$now" >> "$tmp_file" ``` ]]>

SkillSpector

Vulnerability Patterns

Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep

Findings (3)

External Transmission

Medium

Category: Data Exfiltration
Confidence: 60% confidence
Finding: Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.
Content: Scanner excerpt · SKILL.md (reported line 26)May include surrounding context.
md
compatibility: > Requires pilot-protocol skill and pilotctl binary on PATH. The daemon must be running (pilotctl daemon start). Requires jq and curl for webhook delivery. metadata: author: vulture-labs version: "1.0"

Missing User Warnings

Medium

Category: Not specified by scanner
Confidence: 94% confidence
Finding: The skill includes a concrete workflow that forwards alert contents to an external Slack webhook, but it does not warn users that event payloads may contain sensitive operational data, secrets, or customer information. In an alerting skill, forwarding data externally is expected behavior, but the lack of disclosure minimization and user-facing caution makes accidental data leakage plausible.
Content: No source excerpt is available for this finding.

External Transmission

Medium

Category: Data Exfiltration
Confidence: 96% confidence
Finding: This example performs an outbound POST to a user-configured webhook with alert content derived from subscribed events. Because alert messages may contain sensitive incident details or untrusted content, this creates a real exfiltration path to third-party services, especially in environments where events can include secrets, identifiers, or internal state.
Content: Scanner excerpt · SKILL.md (reported line 80)May include surrounding context.
md
# Alert via Slack slack_payload=$(jq -n --arg msg "$message" '{text: "CRITICAL", attachments: [{color: "danger", text: $msg}]}') curl -X POST "$WEBHOOK_URL" -H "Content-Type: application/json" -d "$slack_payload" --silent # Alert via Pilot pilotctl --json send-message oncall-agent --data "{\"type\":\"critical_alert\",\"message\":\"$message\"}"

Static analysis

No suspicious patterns detected.