Security audit

autonomous-science-loop

Security checks for vulnerabilities and agentic risk

Overview

The science-discovery tool itself is limited, but the bundled self-learning system persists user/workflow data and instructs future modification of the skill file without clear consent or retention limits.

Review this before installing. The science-loop code is local and does not show network exfiltration, but the skill asks the agent to keep a persistent usage/preference log and to evolve the skill instructions over time. Install only if you are comfortable with local cross-session memory and self-modifying skill guidance, or remove/disable learner.py and the self-evolution section first.

SkillSpector

Vulnerability Patterns

Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection

Findings (11)

Lp3

Medium

Category: MCP Least Privilege
Confidence: 93% confidence
Finding: The skill documents file-writing behavior (`learned_patterns.json` and later rewriting `SKILL.md`) but does not declare any permissions or clearly constrain those write targets. Undeclared write capability weakens reviewability and can enable unexpected persistence or tampering, especially when the skill also claims to 'automatically' learn and update itself.

Tp4

High

Category: MCP Tool Poisoning
Confidence: 97% confidence
Finding: The stated purpose is scientific law discovery, but the documentation adds a separate persistence and self-learning subsystem that records usage, failures, notes, and preferences across sessions. This mismatch is dangerous because reviewers and users may trust the skill for one bounded purpose while it silently accumulates data and evolves behavior outside that scope.

Description-Behavior Mismatch

High

Confidence: 98% confidence
Finding: The skill expands from a science-discovery tool into a persistent self-learning system that stores operational history and user preferences. That creates hidden state and cross-session behavior unrelated to the core capability, increasing privacy risk and making outputs less predictable and harder to audit.

Context-Inappropriate Capability

Medium

Confidence: 95% confidence
Finding: Persisting user preferences and error history is not necessary for fitting hypotheses or designing experiments, so the data collection exceeds functional need. Unnecessary retention increases privacy exposure and can accumulate sensitive operational context without clear justification.

Context-Inappropriate Capability

High

Confidence: 99% confidence
Finding: The instruction to write experience back into `SKILL.md` introduces self-modifying behavior in the skill definition itself. Self-modification is especially risky because it can alter future behavior, bypass prior review assumptions, and create a pathway for prompt/instruction drift or persistence of unsafe content.

Description-Behavior Mismatch

High

Confidence: 96% confidence
Finding: The file implements a generic telemetry and preference logging component for arbitrary skills, which materially exceeds the declared purpose of an autonomous scientific discovery loop. This creates an unjustified cross-skill data collection primitive that can persist operational history and user-related metadata, increasing privacy and abuse risk if reused broadly.

Context-Inappropriate Capability

Medium

Confidence: 94% confidence
Finding: The code stores arbitrary preferences, notes, capabilities, and recent operation history in a persistent JSON file without purpose limitation or data minimization. In the context of a science-discovery skill, this is dangerous because free-form notes and preferences can accumulate sensitive user or workflow data unrelated to the advertised function.

Intent-Code Divergence

Medium

Confidence: 87% confidence
Finding: The documentation explicitly describes the module as skill-agnostic self-evolution infrastructure for any WorkBuddy skill, contradicting the narrowly scoped skill description. This mismatch is risky because disguised general-purpose infrastructure can be used to expand data collection or behavior beyond what users and reviewers expect.

Missing User Warnings

Medium

Confidence: 94% confidence
Finding: The documentation says usage data and user preferences are automatically persisted, but it does not provide a clear privacy warning, consent mechanism, or retention policy. Silent cross-session storage can surprise users and expose personal or operational information beyond what they intended to share.

Missing User Warnings

Medium

Confidence: 92% confidence
Finding: The instructions encourage modifying `SKILL.md` without clearly warning that a skill file is being changed. This is dangerous because users or operators may not realize they are authorizing persistent changes to the agent's definition, creating integrity and maintainability risks.

Ssd 3

Medium

Confidence: 96% confidence
Finding: The natural-language instructions explicitly direct retention of user preferences, usage details, and error notes in a local memory file across sessions. Cross-session memory can capture sensitive details, influence future behavior in opaque ways, and create a persistent data trail unrelated to the main scientific function.

Static analysis

No suspicious patterns detected.