Back to skill

Security audit

Analyst Watchdog

Security checks for vulnerabilities and agentic risk

Overview

The skill is a coherent local monitoring watchdog, but its implementation lets untrusted localhost API data influence agent-consumed files and can read JSON files outside its intended score directory.

Install only if you control the localhost service on port 8765 and are comfortable with unattended local file writes. Before scheduled use, constrain the output directories, validate API response fields, restrict score keys to safe identifiers, and ensure downstream agents treat generated Markdown as untrusted data.

Vulnerability Patterns
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (2)

T05 · Unauthorized Access and Privilege Escalation

Warning
Location
scripts/analyst_agent.py:145
Finding
API-Controlled Path Traversal Allows Unauthorized Local JSON File Reads<![CDATA[ ## Vulnerability Details **File Location**: `scripts/analyst_agent.py`, lines 145–151 **Vulnerability Type**: Path traversal through an unvalidated API-controlled filename **Risk Level**: Medium ### Vulnerable Code ```python scores_dir = WORKSPACE / "projects" / "hybrid-control-plane" / "data" / "scores" score_file = scores_dir / f"{key}.json" if score_file.exists(): try: with open(score_file) as f: records = json.load(f) return [round(r["score"], 4) for r in records[-10:]] ``` ### Technical Analysis The `key` value originates from a key in the `confidence` object returned by the local `/status` API endpoint. It is passed to `_get_last_10_scores()` and incorporated directly into a filesystem path without validating its characters or verifying that the resolved path remains inside the intended score directory. Python `pathlib` path joining does not prevent traversal. A value containing `../` components can escape the score directory, while an absolute value can cause the preceding base path to be discarded. The script then appends `.json`, opens the resulting path, parses it, and copies up to ten `score` values into `FINDINGS.md`. Successful exploitation requires the targeted file to be readable by the process, have a `.json` suffix under the constructed path, contain valid JSON, and have the expected list-of-records structure with `score` fields. ### Attack Path 1. An attacker compromises, impersonates, or otherwise controls the unauthenticated service listening on `localhost:8765`. 2. The attacker returns a `/status` response whose `confidence` object contains a crafted key with traversal components or an absolute pathname. 3. The attacker supplies values that cause `check_milestones()` to generate a milestone event for that key. 4. `update_findings()` passes the attacker-controlled key to `_get_last_10_scores()`. 5. `_get_last_10_scores()` resolves the crafted value without containment validation and opens the ...[truncated 693 chars]
Remediation
<![CDATA[ ## Remediation Suggestions - Restrict score keys to a strict identifier allowlist, such as letters, digits, underscores, and hyphens. - Reject absolute paths, path separators, `.` components, and `..` components. - Resolve both the score directory and candidate path, then verify containment before opening the file. - Validate the loaded JSON against an explicit schema before processing it. - Authenticate the local API or otherwise verify that responses originate from the expected control-plane service. - Run the agent under a dedicated account with access only to the required workspace files. Example containment hardening: ```python import re def _get_last_10_scores(status: dict, key: str) -> list: if not re.fullmatch(r"[A-Za-z0-9_-]+", key): log("WARNING: Rejected invalid score key") return [] scores_dir = ( WORKSPACE / "projects" / "hybrid-control-plane" / "data" / "scores" ).resolve() score_file = (scores_dir / f"{key}.json").resolve() if score_file.parent != scores_dir: log("WARNING: Rejected score path outside score directory") return [] try: with score_file.open() as f: records = json.load(f) if not isinstance(records, list): return [] return [ round(float(record["score"]), 4) for record in records[-10:] if isinstance(record, dict) and "score" in record ] except (FileNotFoundError, OSError, ValueError, TypeError, json.JSONDecodeError): return [] ``` ]]>

T01 · Skill Instruction Hijacking

Warning
Location
scripts/analyst_agent.py:205
Finding
Untrusted Local API Content Is Injected into Agent and Alert Channels<![CDATA[ ## Vulnerability Details **File Location**: `scripts/analyst_agent.py`, lines 205–210; additional sinks at lines 222–225, 249–253, and 261–267 **Vulnerability Type**: Indirect instruction and Markdown injection through untrusted API responses **Risk Level**: Medium ### Vulnerable Code ```python for ev in milestone_events: lines.append(f"- **Milestone**: {ev['model']} {ev['task']} hit n={ev['milestone_n']} (current n={ev['n']}, mean={ev['mean']}, last10={ev['last_10_mean']})") for ev in promotion_events: lines.append(f"- **Promotion**: {ev['key']} promoted — {ev['info']}") ``` The same untrusted event data is also written to the Telegram alert file: ```python lines = [f"🏆 **Model Promotion Alert** — {ts}\n"] for ev in promotion_events: lines.append(f"• {ev['key']}: {ev['info']}") lines.append("") with open(ALERT_PATH, "w") as f: f.write("\n".join(lines) + "\n") ``` Promotion-track values returned by `/trends/summary` are similarly formatted without escaping: ```python lines.append( f"🔥 IMMINENT: {m['model']}/{m['task']} needs only {m['runs_to_promotion']} " f"more runs (mean={m['mean']:.3f}) — promote soon!" ) ``` ```python lines.append( f"- {m['model']}/{m['task']}: mean={m['mean']:.3f} n={m['n']} " f"→ needs {m['runs_to_promotion']} more runs {arrow}" ) ``` ### Technical Analysis Fields including promotion keys, promotion information, model names, and task names originate from HTTP responses and are inserted verbatim into Markdown files. No schema validation, length restriction, Markdown escaping, or trust-boundary annotation is applied. `SKILL.md` states that `OUTBOX.md` is read by an orchestrator and that `ALERT_TELEGRAM.md` is sent immediately. Consequently, attacker-controlled API text can enter channels that downstream agents or users may regard as trusted. Crafted values could add headings, links, mentions, misleading alerts, or natural-language instructions. If an orchestrator places the file conte ...[truncated 1618 chars]
Remediation
<![CDATA[ ## Remediation Suggestions - Authenticate the localhost API and verify that responses originate from the expected service. - Apply explicit schemas to every endpoint response, including strict types, bounded lengths, numeric ranges, and identifier patterns. - Restrict model, task, and event keys to a conservative character allowlist. - Escape Markdown metacharacters before placing untrusted values into human-facing files. - Store machine-consumed events in a structured format such as JSON rather than embedding them directly in free-form Markdown. - Mark all API-derived values as untrusted data when constructing prompts for downstream agents. - Configure the orchestrator never to treat instructions found in report fields as executable instructions. - Separate trusted report templates from untrusted values and include provenance metadata for each event. - Use atomic writes and appropriate filesystem permissions to prevent unrelated local users from modifying generated alert files. A suitable validation layer should reject unexpected fields and cap text lengths before formatting. For example, identifiers could be limited to a pattern such as `^[A-Za-z0-9_.-]{1,100}$`, while descriptive fields should be escaped and treated only as display text. ]]>
Vulnerability Patterns
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
Findings (3)

Lp3

Medium
Category
MCP Least Privilege
Confidence
94% confidence
Finding
The skill advertises operational behavior that includes local file writes and polling a localhost API, but it does not declare an explicit tool/permission scope. That mismatch weakens least-privilege controls and can cause an orchestrator or reviewer to underestimate what the skill is allowed to do, especially for an autonomous scheduled agent.

Vague Triggers

Medium
Confidence
93% confidence
Finding
The description uses broad activation language like 'use when you need an autonomous agent that monitors a system and produces structured analysis,' which can cause the skill to be invoked in situations beyond the author's intended scope. For a skill that runs autonomously and writes alerts/files, overbroad matching increases the chance of unintended execution and unexpected state changes on the host.

Missing User Warnings

Medium
Confidence
92% confidence
Finding
The markdown describes autonomous scheduled file writes and alert generation, including an urgent alert file that is sent and deleted, but it does not prominently warn users that the skill will modify local files without ongoing interaction. In an autonomous context, undisclosed persistence and file mutation are risky because they can surprise users, overwrite artifacts, or create covert signaling channels between agents/processes.

Static analysis

No suspicious patterns detected.