Security audit

Fellow

Security checks for vulnerabilities and agentic risk

Overview

This experiment skill is mostly coherent, but it includes a silent daily self-update that can overwrite its own files from GitHub, so it should be reviewed before installation.

Install only if you are comfortable with this skill creating persistent experiment records and registering a daily background update that can replace the skill from GitHub. Prefer a version with self-update removed or changed to pinned, verified, explicit operator-approved updates.

A.I.G

Vulnerability Patterns

Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs

SkillSpector

Vulnerability Patterns

Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
Rogue AgentSelf-Modification, Session Persistence
Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands

Findings (12)

Self-Modification

High

Category: Rogue Agent
Content: # Task declared in SKILL.md frontmatter metadata.{platform}.cron ``` ## Self-update `fellow.update` pulls the latest package from the `source:` URL in this file's frontmatter. Runs silently — no output unless the version changed or an error occurred. See `references/schemas.md` → **Self-update procedure** for the full step-by-step.
Confidence: 97% confidence
Finding: This section defines built-in self-modification behavior: the skill can update its own package from a remote repository. Self-modifying behavior is high risk because it changes the trusted execution base at runtime and creates a direct path for supply-chain attacks, malicious updates, or policy drift.

Self-Modification

High

Category: Rogue Agent
Content: ## Self-update `fellow.update` pulls the latest package from the `source:` URL in this file's frontmatter. Runs silently — no output unless the version changed or an error occurred. See `references/schemas.md` → **Self-update procedure** for the full step-by-step. ## Visibility
Confidence: 98% confidence
Finding: The specific statement that `fellow.update` pulls the latest package from the `source:` URL confirms active remote code replacement behavior. Because this skill is an internal experimentation engine with authority to evaluate and promote changes, compromise of its update path could affect downstream skills, experiment outcomes, and trusted promotion decisions.

Self-Modification

High

Category: Rogue Agent
Content: --- ## Self-update procedure `fellow.update` pulls the latest package from the `source:` URL in this file's frontmatter.
Confidence: 98% confidence
Finding: Self-modification is a high-risk capability because it allows the skill to alter its own code and behavior outside its primary mission. In this context, an experiment engine should not need autonomous update authority, so the capability materially increases the attack surface and the blast radius of any compromise.

Description-Behavior Mismatch

High

Confidence: 98% confidence
Finding: The schema file for an experimentation engine documents a self-update workflow that fetches and installs remote code, which exceeds the stated scope of benchmark-driven evaluation. This creates an unnecessary software supply-chain and self-modification pathway that could replace local files with attacker-controlled content if the source or update path is compromised.

Context-Inappropriate Capability

High

Confidence: 99% confidence
Finding: The documented procedure downloads a remote tarball from a repository and recursively copies its contents over the local skill directory, effectively granting remote overwrite capability. Because there is no integrity validation, pinning, or sandboxing, compromise of the repository, branch, or transport context could lead to arbitrary code or prompt replacement and full takeover of the skill behavior.

Shadow Command Trigger

Medium

Category: Trigger Abuse
Confidence: 70% confidence
Finding: Shadow Command Trigger: 'run experiments' conflicts with built-in command 'run'

Missing User Warnings

Medium

Confidence: 94% confidence
Finding: The skill explicitly permits pulling and replacing its own package from a remote source and states that the operation runs silently unless a version change or error occurs. A silent remote self-update is dangerous because it can introduce unreviewed code, reduce operator visibility, and enable supply-chain compromise or unauthorized behavior changes without timely detection.

Description-Behavior Mismatch

Medium

Confidence: 95% confidence
Finding: The interactive menu exposes actions that exceed the skill's declared experimentation role, including journal writing and repository updating. This expands the reachable capability surface for anyone invoking the menu and can enable unintended state changes or privileged operations that are not justified by the stated purpose of the skill.

Context-Inappropriate Capability

Medium

Confidence: 98% confidence
Finding: Including an 'update — Pull latest from GitHub' action gives an experimentation skill a code- or configuration-changing capability unrelated to benchmark execution. If invoked improperly, this can introduce unreviewed changes, supply-chain risk, or unauthorized modification of the local environment, especially because the menu makes the action directly discoverable in interactive use.

Missing User Warnings

Medium

Confidence: 88% confidence
Finding: The skill mandates persistent journal creation for every run, including command text, timing, runtime metadata, and decision details, but provides no requirement to notify users or minimize sensitive content before storage. In an experimentation engine, this can silently retain potentially sensitive prompts, benchmark inputs, or operational metadata and create privacy, compliance, or data-handling risks if users or upstream callers are unaware of the logging behavior.

Intent-Code Divergence

Medium

Confidence: 84% confidence
Finding: The header says this file is the canonical source for experiment invocation contracts, cycle outputs, configuration defaults, and OKR definitions. Later, the file introduces a `Self-update procedure` that performs remote fetch and local installation, which contradicts the documented scope of the file rather than merely extending one of the listed schema areas.

Missing User Warnings

Medium

Confidence: 96% confidence
Finding: The self-update instructions describe system-modifying behavior that downloads and overwrites local files without clearly surfacing the risk or requiring user acknowledgement. In practice, this increases the chance of unsafe execution by operators or automation and hides a highly privileged action inside what appears to be ordinary reference documentation.

Static analysis

No suspicious patterns detected.