Back to skill

Security audit

hypothesis-driven-inquiry

Security checks for vulnerabilities and agentic risk

Overview

The core reasoning tool is local, but the skill also includes persistent learning and preference storage that can influence later use without clear limits or deletion controls.

Review the learning feature before installing. Use the core hypothesis script only if you do not want persistent state, avoid recording secrets or sensitive incident details in preferences or notes, and periodically inspect or delete learned_patterns.json. Stored preferences should be treated as untrusted data and should not override the current user request or higher-priority instructions.

Vulnerability Patterns
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (2)

T02 · Agent Memory Poisoning

Warning
Location
scripts/learner.py:94
Finding

Unvalidated Persistent Preferences Can Poison Future Agent Sessions

Content
View full analysis

Vulnerability Details

File Location: scripts/learner.py:94-98; related behavioral guidance in SKILL.md:54-67
Vulnerability Type: Persistent storage of untrusted behavioral preferences
Risk Level: Medium

Vulnerable Code:

python
def record_preference(skill_dir, key, value):
    data = load_learning(skill_dir)
    data["preferences"][key] = value
    save_learning(skill_dir, data)
    return data

The corresponding command-line interface accepts arbitrary preference keys and values:

python
p_pre  = sub.add_parser("prefer"); p_pre.add_argument("skill_dir")
p_pre.add_argument("--key", required=True); p_pre.add_argument("--val", required=True)

Technical Analysis

The prefer operation accepts arbitrary command-line strings and stores them directly in the persistent preferences object inside learned_patterns.json. There is no allowlist of supported preference keys, value validation, length restriction, provenance tracking, trust classification, or user-confirmation mechanism.

The Skill documentation instructs subsequent uses to adopt important preferences from this persistent file. Consequently, a value containing adversarial instructions can cross the boundary between passive stored data and future Agent guidance. The vulnerability becomes exploitable when attacker-controlled or prompt-injected content is passed to the prefer operation and a later Agent invocation interprets the stored value as an instruction.

The code does not itself execute the stored text as Python or a shell command. The risk is persistent manipulation of Agent behavior rather than direct native code execution.

Attack Path

  1. An attacker supplies content that is treated as a user preference, either directly or through prompt injection in data processed by the Agent.
  2. The Agent or operator invokes:
    bash
    python scripts/learner.py prefer /path/to/skill \
      --key behavior \
    

...[truncated 1085 chars]

Remediation
View remediation

Remediation Suggestions

  1. Define an explicit allowlist of supported preference keys.
  2. Enforce type, format, length, and character restrictions for every allowed value.
  3. Reject values containing instruction-like content when only declarative settings are expected.
  4. Store provenance metadata, including who supplied the value, when it was recorded, and whether the user explicitly approved persistence.
  5. Require clear user confirmation before writing any behavioral preference.
  6. Treat all loaded preferences as untrusted data rather than executable or authoritative instructions.
  7. Ensure stored preferences can never override system instructions, developer instructions, safety controls, or the current user's explicit request.
  8. Provide commands to review, revoke, and reset persisted preferences.
  9. Separate passive analytics from behavioral configuration so usage records cannot become Agent instructions.
  10. Add tests proving that unknown keys and instruction-like values are rejected.

T09 · Insecure Skill Coding Practices

Note
Location
scripts/hypothesis.py:23
Finding

Unvalidated Complexity Values Can Crash or Corrupt Hypothesis Ranking

Content
View full analysis

Vulnerability Details

File Location: scripts/hypothesis.py:23-27, 36
Vulnerability Type: Improper numeric input validation
Risk Level: Low

Vulnerable Code:

python
        elif part.startswith("complexity="):
            try:
                d["complexity"] = float(part[len("complexity="):])
            except Exception:
                d["complexity"] = 1
python
        parsimony = 1.0 / (1.0 + h["complexity"])

Technical Analysis

The parser accepts any string that Python's float() function recognizes. It does not enforce that complexity is finite, nonnegative, or within a reasonable range.

A complexity value of -1 makes the denominator in the parsimony calculation equal to zero:

text
1 + (-1) = 0

The subsequent division raises ZeroDivisionError, terminating the program. Other values such as nan, positive or negative infinity, and values below -1 can produce undefined, non-finite, negative, or misleading scores and therefore corrupt the ranking output.

The exception handler only protects the string-to-float conversion. It does not protect the later arithmetic or validate successfully parsed but unsafe numeric values.

Attack Path

  1. An attacker or untrusted caller controls a --hypo command-line argument.
  2. The caller supplies a complexity value of -1:
    bash
    python scripts/hypothesis.py \
      --obs o1 \
      --hypo "h1|explains=o1|complexity=-1"
    
  3. float("-1") succeeds, so the parser stores -1.0.
  4. rank() evaluates:
    python
    1.0 / (1.0 + -1.0)
    
  5. Python raises ZeroDivisionError, and the process exits without producing a result.

An attacker can alternatively supply nan or infinity to produce non-finite scores and unreliable ordering without necessarily causing an immediate exception.

Impact Assessment

The direct impact is denial of service for the current local invocat ...[truncated 388 chars]

Remediation
View remediation

Remediation Suggestions

Validate complexity immediately after parsing and reject unsafe values before ranking:

python
import math

raw = part[len("complexity="):]
try:
    complexity = float(raw)
except ValueError as exc:
    raise ValueError("Complexity must be a valid number") from exc

if not math.isfinite(complexity) or complexity < 0:
    raise ValueError("Complexity must be a finite, nonnegative number")

d["complexity"] = complexity

In addition:

  1. Return a nonzero exit status and a clear error message for invalid input instead of silently replacing malformed values with 1.
  2. Consider enforcing a reasonable upper bound if extreme complexity values are not meaningful.
  3. Validate dictionary inputs inside rank() as a defense-in-depth measure, because callers can invoke it without using parse_hypo().
  4. Add regression tests for -1, values below zero, nan, positive and negative infinity, malformed strings, and extremely large values.
  5. Catch validation errors in main() and report them without exposing a traceback to ordinary users.
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
Findings (13)

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The documented purpose is local hypothesis-driven reasoning, but the described/observed behavior includes persistent telemetry, preference storage, and reflective learning unrelated to the core task. This mismatch is dangerous because users and orchestration systems may route sensitive analytical tasks to the skill without expecting cross-session storage, creating stealthy data retention and trust-boundary violations.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

High
Category
Not specified by scanner
Confidence
97% confidence
Finding

The skill manifest describes a zero-dependency, locally runnable abduction workflow that infers causes from observations, ranks hypotheses, and designs discriminative experiments with traceable output. This file instead implements a generic telemetry/learning subsystem that records usage, stores user preferences, accumulates error patterns, and suggests skill optimization across arbitrary WorkBuddy skills, which is a materially different function than hypothesis-driven inquiry itself.

Content

No source excerpt is available for this finding.

Undeclared Tool Scope

Medium
Category
MCP Least Privilege
Confidence
90% confidence
Finding

The skill documents file-writing behavior via a learner subsystem but declares no tool scope or permissions boundary. That creates an undeclared capability gap: an operator may invoke a skill believed to be reasoning-only while it persists data locally, which can lead to unauthorized local state changes or retention of sensitive information.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

Medium
Category
Not specified by scanner
Confidence
90% confidence
Finding

Claiming the skill is 'zero-dependency, locally runnable, output traceable' while also maintaining persistent learned state is misleading about its operational and privacy properties. Persistent state changes the trust model because outputs may be influenced by prior sessions and local data may accumulate without the operator appreciating that retention is occurring.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

Recording user preferences and long-term usage telemetry is not necessary for a local abduction/reasoning utility and expands the data collection surface beyond stated purpose. Unnecessary collection increases privacy risk and can capture sensitive habits, project details, or analytical context that should not persist beyond the task.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

The skill describes persistence of user preferences and usage history without warning users about retention or privacy consequences. In practice, this can cause silent storage of sensitive workflow details, especially dangerous because the skill presents itself as a reasoning aid rather than a memory-collecting component.

Content

No source excerpt is available for this finding.

Ssd 3

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The natural-language instructions explicitly direct retention of user preferences across sessions in a local memory file, creating unauthorized or unexpected long-term state. In the context of an analysis skill, this is more dangerous because user inputs may contain sensitive incident details, hypotheses, operational errors, or investigative priorities that should not persist by default.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
90% confidence
Finding

The module description, CLI help text, and user-facing output strings are entirely in Chinese, which effectively imposes a language choice on users without opt-in. The policy allows locale constraints only when explicitly documented and justified or when the user is offered a choice, neither of which appears here.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

The code writes preferences, free-form notes, error metadata, and recent operation history into a persistent JSON file without any user-facing disclosure or consent mechanism. Because notes are arbitrary text, users may inadvertently store secrets, personal information, or sensitive task details, creating a privacy and local information exposure risk.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

The module persistently records user preferences, usage history, error notes, and operational analytics to disk even though that behavior is unrelated to the stated hypothesis-driven inquiry function. In a skill context, this creates unnecessary data collection and retention risk: notes and error strings may contain sensitive user content, and the persistent cross-session profile can be accessed later by other local processes or users.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
90% confidence
Finding

The description and operational guidance are presented in Chinese, which can impose a specific language on users without indicating any option to choose another language. The file does not document that the locale is intentional for a region-specific purpose or provide an opt-in/alternative language.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Low
Category
Not specified by scanner
Confidence
95% confidence
Finding

L50 states that after each use the skill 'automatically' performs review and accumulates experience, implying built-in autonomous learning. But L55-L59 instruct the user to explicitly run separate Bash commands to record, inspect, and reflect, which contradicts the claim of automatic post-use learning.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
95% confidence
Finding

This Python file contains natural-language instructions and descriptions entirely in Chinese, including the main module description and later CLI usage/output strings, but provides no opt-in or alternative language. Under the policy rule for language/locale constraints, this is a locale-specific behavior that is not explained as region-specific or optional.

Content

No source excerpt is available for this finding.