Back to skill

Security audit

cross-model-knowledge-extraction

Security checks for vulnerabilities and agentic risk

Overview

The skill’s main extractor is local and understandable, but it adds persistent learning and self-update guidance that goes beyond the stated extraction purpose.

Review before installing. The extractor itself is a local standard-library tool, but the bundled learner stores usage history, errors, notes, and preferences across runs and encourages future behavior changes. Install only if you are comfortable with local persistent state in the skill directory, and avoid recording secrets, confidential project details, or untrusted instruction-like text in notes or preferences.

Vulnerability Patterns
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
Findings (1)

T02 · Agent Memory Poisoning

Warning
Location
scripts/learner.py:85
Finding

Unvalidated Persistent State Can Influence Future Agent Behavior

Content
View full analysis

Vulnerability Details

File Location: scripts/learner.py:85-96; related behavioral instructions in SKILL.md:70-91 and scripts/section.md:8-29
Vulnerability Type: Persistent agent-memory poisoning
Risk Level: Medium

Vulnerable Code

python
def record_preference(skill_dir, key, value):
    data = load_learning(skill_dir)
    data["preferences"][key] = value
    save_learning(skill_dir, data)
    return data

def get_insights(skill_dir):
    data = load_learning(skill_dir)
    caps = sorted(data["capabilityStats"].items(), key=lambda kv: kv[1]["count"], reverse=True)
    errors = sorted(data["errorPatterns"].items(), key=lambda kv: kv[1]["count"], reverse=True)
    return {
        "totalOps": data["totalOps"],
        "topCapabilities": [(k, v["count"]) for k, v in caps[:5]],
        "recurringErrors": [(k, v["count"]) for k, v in errors[:5]],
        "preferences": data["preferences"],
    }

The same persistent-state mechanism also accepts arbitrary error names and notes:

python
if error:
    ep = data["errorPatterns"].setdefault(error, {"count": 0, "lastNote": "", "lastTime": ""})
    ep["count"] += 1
    ep["lastNote"] = note or ""
    ep["lastTime"] = data["lastUpdated"] or datetime.datetime.now().strftime("%Y-%m-%dT%H:%M")
rec = {"t": data["lastUpdated"], "cap": capability, "ok": success, "err": error, "note": note}
data["recentOps"].append(rec)
data["recentOps"] = data["recentOps"][-20:]
save_learning(skill_dir, data)

Technical Analysis

The learning subsystem writes caller-controlled preference keys, preference values, capability names, error labels, and notes to learned_patterns.json without schema restrictions, trust metadata, content validation, expiration, or user confirmation.

The instructions in SKILL.md:70-91 and the duplicate template in scripts/section.md:8-29 direct the Agent to retain important user preferences, apply them ...[truncated 2115 chars]

Remediation
View remediation

Remediation Suggestions

  1. Remove the generic learning subsystem from this extraction-only Skill unless persistent personalization is an explicit requirement.
  2. Make all persistent learning opt-in and obtain clear user confirmation before writing preferences, notes, or error details.
  3. Define an allowlisted schema for preference keys and constrain values by type, length, and permitted character set.
  4. Do not store arbitrary instruction-like free text. Treat all stored observations as untrusted data rather than Agent instructions.
  5. Add provenance fields recording the source, user, task, creation time, trust level, and expiration time for every stored item.
  6. Isolate state by user and workspace to prevent one user or task from influencing another.
  7. Never rewrite SKILL.md automatically from accumulated state. Require an explicit, human-reviewed change with a visible diff.
  8. Add retention limits and deletion controls, and warn users not to place credentials, personal data, or confidential content in notes and preferences.
  9. When presenting insights to an Agent, delimit and escape stored values and explicitly state that they are untrusted observations that must not override system, developer, or current-user instructions.
  10. Use atomic file replacement and restrictive file permissions when saving state to reduce corruption and unintended disclosure risks.
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Behavioral ASTexec() Call, eval() Call, Dynamic Import
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
Findings (22)

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

The documented purpose is local signature extraction, but the skill text also introduces undeclared persistent learning behavior, including recording usage, errors, and user preferences to learned_patterns.json. This mismatch is dangerous because operators may grant trust and access based on the declared extraction-only function while the skill actually performs broader stateful data collection and retention.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

High
Category
Not specified by scanner
Confidence
97% confidence
Finding

The skill expands itself from a one-shot local extraction utility into a persistent self-learning system that stores longitudinal operational data. In the context of a skill expected to process other skill content, this broadening of scope is especially risky because it introduces hidden state, retention, and future behavioral changes unrelated to the user's immediate task.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

High
Category
Not specified by scanner
Confidence
98% confidence
Finding

Persistent collection of user preferences, usage frequency, and error history is not necessary to extract structured capability signatures from markdown text. Because the stated purpose does not justify profile building, the data collection creates unnecessary privacy and governance risk, especially if the notes or errors contain sensitive user or project information.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

High
Category
Not specified by scanner
Confidence
97% confidence
Finding

The file explicitly implements persistent storage of usage history, error telemetry, recent operations, and user preferences in learned_patterns.json, which is unrelated to the stated extraction-only purpose of the skill. In a skill meant for local structured knowledge extraction, this broad telemetry capability increases privacy risk, expands the data-retention surface, and enables silent accumulation of behavioral data without clear necessity or consent boundaries.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

High
Category
Not specified by scanner
Confidence
95% confidence
Finding

The code provides a reusable self-evolution and behavior-tracking subsystem that records operations, preferences, recurring errors, and optimization suggestions for arbitrary skills, not just this extraction skill. That creates an unjustified cross-cutting capability that can be repurposed for surveillance or hidden persistence, making the skill more dangerous because its declared purpose does not require adaptive tracking at all.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

High
Category
Not specified by scanner
Confidence
95% confidence
Finding

The section introduces a persistent self-learning subsystem that records history, preferences, and modifies future behavior, which materially exceeds the manifest’s stated scope of local knowledge extraction to JSON. Scope expansion like this is dangerous because it creates undeclared statefulness and behavior drift, enabling retention of user-derived data and unreviewed changes to how the skill operates over time.

Content

No source excerpt is available for this finding.

Undeclared Tool Scope

Medium
Category
MCP Least Privilege
Confidence
88% confidence
Finding

The skill advertises and instructs use of local scripts that read and write files, but it does not declare any explicit tool scope or permissions boundary. That makes the file access surface implicit and harder for reviewers or runtime policy to constrain, increasing the risk of unintended access or persistence beyond the stated task.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The markdown directs storage of user preferences and usage history but provides no explicit privacy notice, retention policy, or warning about what may be written to disk. This is dangerous because users and operators may unknowingly persist potentially sensitive data in local skill directories without understanding how long it remains or who can access it.

Content

No source excerpt is available for this finding.

Ssd 3

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

The instructions explicitly direct persistent storage of preferences and usage-derived data for future automatic reuse, creating a standing memory of prior interactions. In a skill whose stated role is document extraction, that hidden carryover can leak context across sessions, surprise users, and enable unintended profiling or influence on later outputs.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
88% confidence
Finding

The example command stores 输出语言 with value 中文 and presents it as the direct preference to use next time, while the document does not state that language choice is optional or user-selected. This creates a natural-language locale policy concern because the skill appears to default to a specific language without explicit opt-in or documented justification.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

This code file contains natural-language instructions and runtime error text in Chinese only, including the module docstring and the missing-file message. Because the skill forces a specific language for users without opt-in or justification, it matches the language/locale policy-violation category.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The usage examples shown in the docstring are natural-language interface text and are presented only in Chinese. This can violate language/locale policy when the skill does not offer an alternative language or state a justified region-specific scope.

Content

No source excerpt is available for this finding.

Dynamic import via __import__()

Medium
Category
Dangerous Code Execution
Confidence
75% confidence
Finding

Dynamic import() can load arbitrary modules at runtime, bypassing static analysis and potentially importing malicious code.

Content

Scanner excerpt · scripts/extract_signature.py (reported line 38)May include surrounding context.

python
seen = set(); limits = [x for x in limits if not (x in seen or seen.add(x))]
    # 工具脚本
    scripts = sorted(os.path.basename(s) for s in
                     __import__("glob").glob(os.path.join(skill_dir, "scripts", "*.py")))
    # 可蒸馏决策规则:含 if/当...时/若/必须/则/should/must 的句子
    sents = re.split(r"(?<=[。!?\n])", body)
    rule_kw = re.compile(r"(?i)(若|如果|当.{0,8}时|则|必须|should|must|if\b.+then|除非|否则)")

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
96% confidence
Finding

This code file includes its module docstring, CLI description, usage text, and output messages entirely in Chinese, which imposes a specific language on users without any opt-in or alternative locale handling. The stated policy flags language or locale constraints when the skill does not offer user choice or clearly document a justified regional limitation.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium
Category
Not specified by scanner
Confidence
92% confidence
Finding

Persisting user preferences and operational history is not necessary for a skill whose stated purpose is extracting structured capability signatures from skill text. This mismatch increases risk by collecting extra data unrelated to core function, creating unnecessary retention and a channel for sensitive notes or habits to accumulate across runs.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

The documentation states that learned_patterns.json stores operation counts, error patterns, user preferences, and improvement suggestions, but gives no warning about retention or privacy implications. This is dangerous because users and operators may unknowingly persist potentially sensitive behavioral or free-text information locally, increasing exposure and compliance risk.

Content

No source excerpt is available for this finding.

Ssd 3

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

Persisting preferences, failure reasons, and usage notes across runs creates a natural-language retention channel that can capture sensitive or identifying user information. In this skill context, such data retention is especially unjustified because the declared function is extraction, not longitudinal user modeling or telemetry.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

The example command persists 输出语言 with value 中文, which indicates a fixed language preference being recorded and reused. This is a natural-language policy concern because the skill does not offer a user language choice or document that Chinese is optional.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Medium
Category
Not specified by scanner
Confidence
92% confidence
Finding

The instruction to write learned experience back into SKILL.md turns a read/parse skill into one that mutates its own instruction corpus. That is risky because it enables instruction drift, accidental corruption, and potential prompt or policy poisoning through accumulated untrusted content written into files that may later be treated as authoritative.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
89% confidence
Finding

Telling the system to write learned experience back into SKILL.md without warning means the skill may modify its own files unexpectedly. Silent modification of skill files is dangerous because it undermines integrity, surprises operators, and can introduce persistent changes that are difficult to audit or revert.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
94% confidence
Finding

This JSON file contains natural-language labels such as "知识提取" in capability metadata and recent operation records, but provides no indication that the skill is region-specific or that users can opt into this locale. Under the policy, forcing a specific language without opt-in can be a natural-language policy violation.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
84% confidence
Finding

The self-test emits human-readable diagnostics entirely in Chinese, including success and failure messages. Although lower impact than primary user flows, this still constitutes a natural-language locale restriction in a code file without opt-in or justification.

Content

No source excerpt is available for this finding.