Back to skill

Security audit

AS Autoresearch Loop

Security checks for vulnerabilities and agentic risk

Overview

The main skill is a legitimate improvement-loop helper, but the package also ships skill copies that tell agents to trigger broadly and run forever, so it needs review before installation.

Install only if you want an agent to iteratively edit files, run evaluations, and keep persistent experiment logs. Before using it, remove or quarantine the nested and versioned SKILL copies with unbounded instructions, set hard limits on iterations/time/cost, use a sandbox or branch, redact real user data from evals, and avoid unattended runs against production workflows.

Vulnerability Patterns
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
Findings (1)

T01 · Skill Instruction Hijacking

Error
Location
autoresearch-loop/SKILL.md:4
Finding

Unbounded Autonomous Execution Through Skill Instruction Hijacking

Content
View full analysis

Vulnerability Details

File Location: autoresearch-loop/SKILL.md, lines 4–20, 157, and 311
Vulnerability Type: Unbounded autonomous execution and suppression of consent checkpoints
Risk Level: High

Vulnerable Code

markdown
Apply Karpathy's autoresearch methodology to autonomously iterate and improve anything measurable —
Claude skills, n8n workflows, system prompts, business processes, or any artifact with a clear quality metric.
Inspired by github.com/karpathy/autoresearch (56k stars). The loop: propose a change → test it → measure
against the target metric → keep if better, discard if not → repeat indefinitely.

ALWAYS trigger this skill when the user says things like: "improve this skill automatically", "iterate on this
workflow", "run autoresearch on", "autonomous improvement loop", "overnight experiments", "keep iterating until
it's better", "run experiments on this", "optimize this automatically", or any request to have Claude
autonomously make and test changes to something measurable. Also trigger for requests to "set up an improvement
loop", "run the autoresearch method", or anything involving iterative autonomous experimentation.
markdown
**Core idea**: Define what "better" means. Lock everything except the artifact being improved. Propose a change → test → measure → keep or discard → repeat. Never stop until interrupted.
markdown
**LOOP FOREVER** (until the user manually interrupts):
markdown
**NEVER STOP (mid-session)**: Once the loop starts, do not pause to ask the user if you should continue. The human might be away. Run until manually interrupted. If you run out of ideas, think harder — look at failed test cases, try the inverse of something that worked, try combining near-misses, escalate to a radical rewrite.

Technical Analysis

The Skill uses imperative instructions to alter the Agent's execution policy when loaded. It mandates broad automatic activation, indefinite iteration, and avoida ...[truncated 2564 chars]

Remediation
View remediation

Remediation Suggestions

  1. Replace all indefinite-loop directives with mandatory bounded execution. Require the setup to define:

    • Maximum iteration count.
    • Maximum wall-clock duration.
    • Maximum model or tool-call count.
    • Maximum cost or external-integration budget.
    • Explicit convergence and error thresholds.
  2. Replace ALWAYS trigger with narrowly scoped activation language based on explicit user intent. Do not activate merely because a request loosely relates to optimization.

  3. Remove instructions that suppress user interaction. Require renewed confirmation:

    • Before extending any established limit.
    • Before executing costly or externally visible operations.
    • Before modifying production artifacts.
    • Before changing the agreed artifact scope or allowed paths.
  4. Add hard safety termination conditions for repeated failures, evaluation errors, unavailable users, rate limits, anomalous costs, and unexpected external side effects.

  5. Preserve higher-priority instructions explicitly. State that the loop must never override system policies, user-imposed limits, tool authorization requirements, or platform safety controls.

  6. Reconcile the duplicate active files so the root and nested SKILL.md define the same bounded behavior. Remove unsafe historical copies from Skill discovery paths or clearly mark them as non-loadable archives.

  7. Add regression checks that reject phrases such as repeat indefinitely, LOOP FOREVER, Never stop until interrupted, and do not pause to ask the user from future active versions.

Vulnerability Patterns
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
Findings (306)

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 41)May include surrounding context.

md
- A `SKILL.md` file

Vague Triggers

High
Category
Not specified by scanner
Confidence
96% confidence
Finding

The trigger instructions are unusually broad and say to ALWAYS activate for a wide class of common optimization requests. That can cause the agent to enter an autonomous modification loop in situations where the user only wanted normal advice, increasing the chance of unintended file changes, long-running activity, and bypass of safer, narrower workflows.

Content

No source excerpt is available for this finding.

Vague Triggers

High
Category
Not specified by scanner
Confidence
98% confidence
Finding

The trigger language is extremely broad and instructs the agent to ALWAYS activate on a wide class of optimization-related requests. That can cause unintended invocation in contexts where the user did not explicitly consent to autonomous experimentation or file modification, increasing the chance of unsafe edits, scope creep, and execution of a long-running loop on sensitive artifacts.

Content

No source excerpt is available for this finding.

Vague Triggers

High
Category
Not specified by scanner
Confidence
97% confidence
Finding

The skill instructs the agent to ALWAYS trigger on a broad set of phrases, including vague requests about automatic improvement and experimentation. This can cause unintended invocation on ambiguous user input, leading the agent into autonomous modification behavior the user did not clearly authorize.

Content

No source excerpt is available for this finding.

Vague Triggers

High
Category
Not specified by scanner
Confidence
97% confidence
Finding

The trigger language is extremely broad and includes vague phrases like 'run experiments on this' or 'optimize this automatically,' which can cause the skill to activate in contexts the user did not clearly intend. Because the skill then directs autonomous iterative modification, mistaken invocation can cascade into unsolicited edits, evaluations, and prolonged execution.

Content

No source excerpt is available for this finding.

Vague Triggers

High
Category
Not specified by scanner
Confidence
96% confidence
Finding

The skill uses very broad trigger language such as 'ALWAYS trigger' for many loosely related phrases, which can cause the agent to invoke an autonomous experimentation workflow in situations where the user did not intend sustained modification activity. In context, that is risky because the skill is designed to edit artifacts and continue iterating without further confirmation, amplifying the consequences of a mistaken invocation.

Content

No source excerpt is available for this finding.

Vague Triggers

High
Category
Not specified by scanner
Confidence
98% confidence
Finding

The trigger guidance is excessively broad and uses open-ended phrases like "or any request" around autonomous experimentation, which can cause the skill to activate for ordinary optimization requests without clear user intent for a long-running, file-modifying loop. In this context, over-triggering is dangerous because the skill is empowered to make repeated artifact changes and persist logs, so accidental activation can escalate a benign request into autonomous modification behavior.

Content

No source excerpt is available for this finding.

Vague Triggers

High
Category
Not specified by scanner
Confidence
97% confidence
Finding

The trigger guidance is intentionally expansive and includes generic phrases like requests to 'optimize this automatically' or 'run experiments on this,' which can cause the skill to activate in contexts the user did not clearly scope. Because this skill authorizes autonomous modification and repeated experimentation, accidental invocation can escalate into unintended changes to files, prompts, workflows, or processes without sufficiently specific user consent.

Content

No source excerpt is available for this finding.

Vague Triggers

High
Category
Not specified by scanner
Confidence
97% confidence
Finding

The trigger language is extremely broad and mandatory ('ALWAYS trigger') for common optimization requests, increasing the chance the skill activates in contexts where autonomous experimentation, file mutation, or prolonged looping was not the user's informed intent. In practice this can cause overreach into self-modifying behavior or persistent experimentation on arbitrary artifacts with insufficient consent and scope control.

Content

No source excerpt is available for this finding.

Vague Triggers

High
Category
Not specified by scanner
Confidence
97% confidence
Finding

The trigger guidance is extremely broad and includes catch-all phrasing like any request for autonomous improvement or experimentation. In practice this can cause the skill to activate on ordinary requests and escalate from advisory help to autonomous modification workflows, increasing the likelihood of unintended actions on files, prompts, or workflows.

Content

No source excerpt is available for this finding.

Vague Triggers

High
Category
Not specified by scanner
Confidence
98% confidence
Finding

The trigger language is extremely broad, including vague phrases like requests to optimize, iterate, or run experiments automatically. That increases the chance the skill activates in situations where the user did not intend an autonomous file-modifying loop, which is dangerous because the skill is designed to make repeated changes and evaluations.

Content

No source excerpt is available for this finding.

Missing User Warnings

High
Category
Not specified by scanner
Confidence
97% confidence
Finding

The skill promotes autonomous iterative improvement and logging behavior without a prominent upfront warning that it may modify artifacts repeatedly, create result files, and continue until manually stopped. Users may invoke it without understanding that it can persist changes and consume substantial resources.

Content

No source excerpt is available for this finding.

Vague Triggers

High
Category
Not specified by scanner
Confidence
98% confidence
Finding

The trigger guidance is extremely broad and directive ('ALWAYS trigger') for a wide class of phrases, which increases the chance the skill activates when the user did not intend to authorize autonomous experimentation. In this skill's context, unintended invocation is dangerous because activation leads directly to iterative modification, evaluation, and continued execution behavior rather than a bounded advisory response.

Content

No source excerpt is available for this finding.

Vague Triggers

High
Category
Not specified by scanner
Confidence
98% confidence
Finding

The trigger guidance is intentionally expansive and includes vague catch-alls like 'any request' and 'anything involving iterative autonomous experimentation,' which can cause the skill to activate in situations the user did not clearly intend. In this skill, unintended invocation is especially risky because activation leads directly into autonomous modification and repeated experimentation behavior, increasing the chance of unauthorized changes or runaway work.

Content

No source excerpt is available for this finding.

Vague Triggers

High
Category
Not specified by scanner
Confidence
97% confidence
Finding

The trigger language is intentionally expansive, including vague phrases like 'or any request' involving autonomous experimentation. That can cause the skill to activate in contexts the user did not clearly intend, leading an agent to begin autonomous modification or experimentation on artifacts without sufficiently explicit consent or scoping.

Content

No source excerpt is available for this finding.

Vague Triggers

High
Category
Not specified by scanner
Confidence
97% confidence
Finding

The skill uses expansive trigger criteria like 'or any request' and 'ALWAYS trigger,' which can cause the agent to invoke an autonomous modification loop in situations where the user did not intend sustained experimentation or file changes. In a system that can edit artifacts and run repeated evaluations, overbroad activation increases the chance of unintended autonomous actions, cost overruns, and accidental modification of sensitive or production content.

Content

No source excerpt is available for this finding.

Vague Triggers

High
Category
Not specified by scanner
Confidence
98% confidence
Finding

The trigger language is intentionally expansive and includes vague catch-alls like 'any request' and 'anything involving iterative autonomous experimentation,' which can cause the skill to activate in situations the user did not clearly intend. In a system that can modify artifacts and run repeated experiments, accidental invocation can lead to unauthorized changes, wasted compute, and unsafe automation scope expansion.

Content

No source excerpt is available for this finding.

Vague Triggers

High
Category
Not specified by scanner
Confidence
98% confidence
Finding

The trigger conditions are excessively broad and include catch-all wording such as 'anything involving iterative autonomous experimentation,' which can cause the skill to activate for ordinary optimization requests without clear user intent. In this skill's context, over-triggering is dangerous because activation leads directly into autonomous modification, testing, file operations, and potentially long-running loops.

Content

No source excerpt is available for this finding.

Vague Triggers

High
Category
Not specified by scanner
Confidence
97% confidence
Finding

The trigger guidance is overly broad and includes vague phrases like 'run experiments on this' or 'optimize this automatically,' which can cause the skill to activate in contexts the user did not intend. Because this skill authorizes autonomous experimentation and modification behavior, accidental invocation could lead to unnecessary file changes, repeated runs, or costly actions on the wrong artifact.

Content

No source excerpt is available for this finding.

Vague Triggers

High
Category
Not specified by scanner
Confidence
97% confidence
Finding

The trigger language says to ALWAYS activate for broad phrases like 'optimize this automatically' or 'run experiments on this,' which can sweep in ambiguous requests and launch autonomous modification behavior without clear user intent. Because the skill includes indefinite looping and file writes, over-triggering materially increases the chance of unsafe autonomous actions.

Content

No source excerpt is available for this finding.

Vague Triggers

High
Category
Not specified by scanner
Confidence
98% confidence
Finding

The trigger language is extremely broad and mandatory ('ALWAYS trigger'), covering many loosely related phrases about improvement or experimentation. In practice, this can cause unintended activation of an autonomous file-modifying loop in contexts where the user did not knowingly request persistent changes, increasing the risk of unauthorized edits, excessive actions, or unsafe optimization of sensitive artifacts.

Content

No source excerpt is available for this finding.

Vague Triggers

High
Category
Not specified by scanner
Confidence
97% confidence
Finding

The trigger language is extremely broad and imperative ('ALWAYS trigger') for phrases like 'optimize this automatically' or 'run experiments on this,' which increases the chance the skill activates on ambiguous requests and begins autonomous modification behavior without sufficiently specific user consent. In the context of a skill that can edit artifacts, run evaluations, and continue iterating indefinitely, accidental invocation materially increases the risk of unintended file changes and uncontrolled agent activity.

Content

No source excerpt is available for this finding.

Vague Triggers

High
Category
Not specified by scanner
Confidence
96% confidence
Finding

The trigger language is extremely broad and includes common phrases that could match ordinary requests for improvement, causing the skill to activate in situations where the user did not intend autonomous experimentation or file modification. Because this skill can launch indefinite iterative loops and modify artifacts, over-triggering materially raises the risk of unintended autonomous behavior.

Content

No source excerpt is available for this finding.

Vague Triggers

High
Category
Not specified by scanner
Confidence
96% confidence
Finding

The trigger guidance is overly broad and includes catch-all phrases like requests to 'optimize this automatically' or 'anything involving iterative autonomous experimentation,' which can activate the skill for ordinary improvement requests. In this skill's context, over-triggering is dangerous because activation leads directly into autonomous modification, evaluation, and repeated iteration on user artifacts, increasing the chance of unintended file changes and prolonged autonomous action.

Content

No source excerpt is available for this finding.

Vague Triggers

High
Category
Not specified by scanner
Confidence
97% confidence
Finding

The trigger guidance is extremely broad and uses 'ALWAYS trigger' for a wide range of loosely related phrases, which can cause the skill to activate in situations where the user did not intend autonomous experimentation or file modification. In this skill's context, unintended invocation is especially risky because the body of the skill instructs autonomous iterative changes, persistent logging, backup/revert operations, and potentially long-running loops.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.