Back to skill

Security audit

self-eval

Security checks for vulnerabilities and agentic risk

Overview

The skill can grade answers offline, but it also includes persistent learning code that can write usage and preference data into arbitrary skill directories, so it needs review before installation.

Install only if you are comfortable with a self-evaluation skill retaining local usage history, error notes, and preferences. Treat learner.py as review-required: constrain it to this skill's own directory or avoid using it, especially in environments with multiple installed skills under ~/.workbuddy/skills.

Vulnerability Patterns
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
Vulnerability Patterns
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
Findings (8)

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The declared purpose is a self-evaluation/rubric scorer, but the skill also describes persistent logging, preference storage, insight generation, reflection, and reads/writes under the skill directory and ~/.workbuddy/skills. This mismatch is dangerous because users and higher-level agents may invoke it expecting a narrow scoring function while it performs broader stateful actions that can collect data, alter local state, and influence future behavior without clear disclosure.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

High
Category
Not specified by scanner
Confidence
98% confidence
Finding

The file implements a persistent cross-session learning and preference store for arbitrary skills, which materially exceeds the declared purpose of a self-evaluation/rubric scorer. In an agent ecosystem, accumulating usage history, errors, and user preferences across skills creates an unnecessary statefulness and data-retention surface that can leak context, enable profiling, or influence later behavior outside the user's immediate request.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

High
Category
Not specified by scanner
Confidence
99% confidence
Finding

resolve_skill_dir accepts either any directory path or any skill name under ~/.workbuddy/skills, allowing this script to read and modify learned_patterns.json for other skills. For a self-evaluation tool, arbitrary cross-skill mutation is unjustified privilege and could be abused to tamper with other skills' state, poison preferences, or persist misleading operational history across the agent environment.

Content

No source excerpt is available for this finding.

Undeclared Tool Scope

Medium
Category
MCP Least Privilege
Confidence
86% confidence
Finding

The skill documentation directs the agent to run Python scripts that write reports and record learning data, but the skill declares no explicit tool scope or permissions. In a public skill, this creates an authorization ambiguity where an orchestrator may permit file-writing behavior without the user or platform having clear visibility into what paths and artifacts can be modified.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

The description says the skill should be called when the user asks things like '评估一下这段输出', '给自己的回答打分', or '检查质量'. These phrases are fairly common and broad, and the file does not provide explicit scope boundaries or negative examples to clarify when the skill should not activate.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
87% confidence
Finding

The skill content and examples are written as if the skill operates in Chinese by default, including the title, rubric labels, and command examples, with no statement that users may choose another language. This can violate language/locale policy when a skill implicitly fixes a language without opt-in.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Medium
Category
Not specified by scanner
Confidence
77% confidence
Finding

The docstring says the module is skill-agnostic and 'only responsible' for recording usage, preferences, insights, and automatic retrospection. However, self_reflect generates advice to write experience back into the skill's SKILL.md, which expands the intended effect toward modifying skill documentation rather than merely tracking learning data. That is an intent-level contradiction between documentation and code comments/output guidance.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
93% confidence
Finding

The file’s user-facing documentation, CLI descriptions, and output examples are entirely in Chinese, which imposes a specific language on users. Under the policy, locale or language constraints should be optional or explicitly justified as region-specific; neither is present here.

Content

No source excerpt is available for this finding.