Back to skill

Security audit

πŸ€–πŸ€πŸ§  better collab with your agent

Security checks for vulnerabilities and agentic risk

Overview

The skill mostly performs the advertised local profiling, but it handles very sensitive conversation history and encourages persistent agent personalization without enough safeguards.

Install only if you are comfortable processing your ChatGPT export locally and storing a derived communication profile. Review and redact the output before putting anything into SOUL.md, AGENTS.md, or another persistent agent file, avoid untrusted custom archetype YAML, and do not run the WildChat test script unless you intentionally want remote dataset access.

Vulnerability Patterns
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Insecure DependenciesIntroduces malicious components through unsafe dependency sources
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (2)

T02 Β· Agent Memory Poisoning

Warning
Location
scripts/analyze_profile.py:756
Finding

Untrusted Archetype Configuration Can Poison Persistent Agent Instructions

Content
View full analysis
Dict[str, Any]: """Load custom archetype definitions from YAML.""" try: import yaml with open(filepath, 'r') as f: data = yaml.safe_load(f) return data.get('archetypes', {}) except ImportError: print("Warning: PyYAML not installed. Using default archetypes only.") return {} except Exception as e: print(f"Warning: Could not load custom archetypes: {e}") return {} ``` ```python else: # prompt-snippet snippet = f"""## User Cognitive Profile - **Primary Archetype:** {profile['insights']['primary_mode']} - **Confidence:** {profile['insights']['primary_confidence']} - **Context Switching:** {profile['insights']['context_switching']} ### Communication Style """ for archetype in profile['archetypes']: rec = archetype['recommendations'] snippet += f""" **{archetype['name']}** ({archetype['metrics']['conversation_count']} conversations) - AI Role: {rec['ai_role']} - Style: {rec['communication_style']} - Keywords: {', '.join(archetype['keywords'][:5])} """ ``` The Skill documentation subsequently instructs users to add generated insights to persistent Agent configuration: ```markdown ### 3. Apply to Your Agent Add to your `SOUL.md` or `AGENTS.md`: ``` ### Technical Analysis `yaml.safe_load` prevents YAML from directly constructing arbitrary Python objects, but it does not establish that loaded strings are safe to use as Agent instructions. Custom archetype fields such as `name`, `ai_role`, and `description` can contain arbitrary multiline text. These val ...[truncated 2409 chars]
Remediation
View remediation

T08 Β· Insecure Dependencies

Note
Location
requirements.txt:1
Finding

Dependency Installation Is Not Reproducible or Integrity-Pinned

Content
View full analysis
=1.3.0 numpy>=1.24.0 PyYAML>=6.0 # Optional: For advanced BM25 ranking (script has built-in fallback) # rank-bm25>=0.2.2 ``` The test dependency file similarly uses open version ranges: ```text # Test script requirements # Install with: pip install -r requirements-test.txt # Core skill requirements (from requirements.txt) scikit-learn>=1.3.0 numpy>=1.24.0 PyYAML>=6.0 # WildChat dataset access datasets>=2.14.0 # Optional: For faster streaming # pyarrow>=12.0.0 ``` The documented installation command is: ```bash pip3 install -r requirements.txt ``` ### Technical Analysis All declared dependencies use lower-bound constraints rather than exact reviewed versions. No lock file or package hashes are provided. Consequently, two installations performed at different times can resolve different package versions and transitive dependency graphs. Python package installation can execute build backend logic and installs code that later runs in the user's process. If a future dependency release, transitive package, configured package index, or downloaded artifact is compromised, the documented installation workflow can introduce attacker-controlled code. The audit did not identify a known malicious or typosquatted package among the listed dependency names. The issue is the lack of reproducibility and artifact integrity enforcement, not evidence that the current packages are malicious. ### Attack Path 1. A user follows the README and runs: ```bash pip3 install -r requirements.txt ``` 2. Pip resolves the newest releases satisfying the open lower bounds from the configured package index. 3. A selected direct or transitive dependency has been compromi ...[truncated 1094 chars]
Remediation
View remediation
Vulnerability Patterns
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Rogue AgentSelf-Modification, Session Persistence
Findings (36)

Anti-Refusal Statement

High
Category
Anti-Refusal
Confidence
80% confidence
Finding

Skill instructs the agent to omit warnings, disclaimers, or ethical commentary. Stripping safety caveats hides risk from the user and is a common jailbreak preamble.

Content

Scanner excerpt Β· examples/custom-archetypes.yaml (reported line 97)May include surrounding context.

yaml
ai_role: "Creative Partner"
    recommendations:
      - "Generate multiple diverse options"
      - "Encourage wild ideas without judgment"
      - "Build on user suggestions"

  # Executive Mode: Strategic and decision-focused

Natural-Language Policy Violations

High
Category
Not specified by scanner
Confidence
95% confidence
Finding

The script is explicitly designed to derive cognitive profiles from conversation history, yet there is no indication of consent, notice, or user choice for this profiling activity. Because cognitive profiling infers sensitive behavioral characteristics, the absence of consent and purpose limitation materially elevates privacy and misuse risk.

Content

No source excerpt is available for this finding.

Context Leakage

High
Category
Data Exfiltration
Confidence
75% confidence
Finding

Code or instructions that leak agent conversation context to external services, potentially exposing sensitive user interactions.

Content

Scanner excerpt Β· test_wildchat.py (reported line 55)May include surrounding context.

python
# Load dataset in streaming mode
    dataset = load_dataset("allenai/WildChat-1M", split="train", streaming=True)
    
    # Collect conversations by user
    user_conversations = defaultdict(list)
    users_complete = set()
    total_processed = 0

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
96% confidence
Finding

The README directs users to export and analyze full ChatGPT conversation history, which can contain highly sensitive personal, professional, financial, health, or credential-related information, but it provides no warning about that sensitivity or guidance on minimizing exposure. In the context of a profiling skill that derives cognitive archetypes, the omission is more dangerous because it encourages users to process and potentially persist especially intimate behavioral data without informed consent or handling precautions.

Content

No source excerpt is available for this finding.

Undeclared Tool Scope

Medium
Category
MCP Least Privilege
Confidence
84% confidence
Finding

The skill clearly instructs users to read local exports and write derived profile data and config files, but it declares no explicit tool scope or permissions. This creates a mismatch between documented behavior and security boundaries, increasing the risk that an agent executes file access without clear user understanding or policy enforcement.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

The skill encourages analysis of full ChatGPT export data, which can contain highly sensitive personal, professional, medical, financial, and behavioral information, yet it does not prominently warn that the inferred profile itself may expose private traits. Users may incorrectly treat the output as harmless metadata even though it can reveal durable behavioral and preference patterns.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

The instructions tell users to paste derived cognitive-profile content into SOUL.md or AGENTS.md, which are persistent context files read by future agents and sessions. That can cause long-term disclosure of sensitive inferred traits, communication patterns, and potentially identifying behavioral signals to any downstream agent with access to those files.

Content

No source excerpt is available for this finding.

Session Persistence

Medium
Category
Rogue Agent
Confidence
60% confidence
Finding

Skill establishes unauthorized persistence across sessions via cron jobs, startup scripts, or state files. Session persistence allows an attacker to maintain access beyond the current interaction.

Content

Scanner excerpt Β· SKILL.md (reported line 123)May include surrounding context.

Define Your Own Archetypes

Create ~/.openclaw/my-archetypes.yaml:

yaml
archetypes:

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The export-for-other-agents feature promotes generating prompt snippets for external AI systems without warning that this transmits inferred psychological or behavioral profile data outside the local environment. Even if raw conversations are not shared, the profile summary can still reveal sensitive traits and preferences that may be retained or further processed by third parties.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
89% confidence
Finding

This JSON manifest defines trigger names such as "technical_keywords", "philosophical_question", and "setup_request" without specifying exact matching criteria, scope boundaries, or exclusion conditions. Because these activation conditions are broad and underspecified, the skill could switch modes during ordinary conversation in ways the user did not intend.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
87% confidence
Finding

The methodology explicitly processes conversation content, titles, timestamps, and unique identifiers, all of which can contain sensitive personal or behavioral information. Without an explicit privacy warning, consent notice, minimization guidance, or handling constraints, users may expose private data to profiling in ways they do not reasonably expect.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
80% confidence
Finding

The phrase 'English-language optimized' signals a language-specific bias or constraint. Under the language/locale policy, this should either be presented as an explicit opt-in/choice for users or be more clearly justified as a documented limitation with guidance for non-English use.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The script ingests highly sensitive conversation exports and derives behavioral or cognitive profiles, but it does not provide clear consent, privacy, retention, or downstream-use warnings to the operator. In this skill context, the absence of safeguards is more dangerous because the tool explicitly encourages profiling and later reuse of inferred traits, which can expose intimate personal data and create unauthorized surveillance or manipulation risks.

Content

No source excerpt is available for this finding.

Dynamic import via __import__()

Medium
Category
Dangerous Code Execution
Confidence
75% confidence
Finding

Dynamic import() can load arbitrary modules at runtime, bypassing static analysis and potentially importing malicious code.

Content

Scanner excerpt Β· scripts/analyze_profile.py (reported line 88)May include surrounding context.

python
for term, df in self.df.items():
            # BM25 IDF formula
            self.idf[term] = np.log((self.doc_count - df + 0.5) / (df + 0.5) + 1.0) if NUMPY_AVAILABLE else \
                           __import__('math').log((self.doc_count - df + 0.5) / (df + 0.5) + 1.0)
    
    def get_scores(self, query: str) -> List[float]:
        """

Ssd 3

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

The prompt-snippet output is designed to persist inferred user traits, preferences, and conversation-derived keywords into future agent instructions, effectively transferring sensitive behavioral profiling into downstream contexts. In this skill's context, that materially increases risk because future agents may use those inferences for biased treatment, over-collection, privacy leakage, or manipulative personalization without the user's informed awareness.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

The script fetches and processes the external WildChat-1M dataset, which contains user-linked conversation records keyed by hashed_ip. That expands the skill from analyzing user-provided exports into collecting third-party conversational data, creating a privacy and scope-creep risk that is not justified by the stated skill purpose.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium
Category
Not specified by scanner
Confidence
90% confidence
Finding

The call to load_dataset pulls data from Hugging Face at runtime, introducing remote data access that is not necessary for local analysis of conversation exports. This increases the attack surface and creates unreviewed data acquisition behavior that may surprise operators and violate least-privilege expectations.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

The profile object stores the full hashed user identifier in user_hash and later persists profiles to disk. Even though the identifier is hashed, retaining stable user-linked identifiers alongside sensitive inferred cognitive profiles enables correlation, re-identification attempts, and long-term tracking.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
90% confidence
Finding

The script writes derived cognitive profiles to a JSON file without any privacy notice, minimization, or storage safeguards. Persisting sensitive inferred attributes from conversations increases the risk of secondary use, leakage, and unauthorized access to profiling data.

Content

No source excerpt is available for this finding.

Vague Triggers

Low
Category
Not specified by scanner
Confidence
78% confidence
Finding

This YAML manifest defines archetype activation using very general keywords such as 'analyze', 'fix', 'learn', 'explain', 'idea', and 'goal' across multiple modes. The file does not document how these keywords are scoped, prioritized, or excluded, which could cause unintended activation during ordinary conversation.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Low
Category
Not specified by scanner
Confidence
80% confidence
Finding

The documentation states under current limitations that the tool does not capture emotional state or sentiment. Later, the same file frames sentiment analysis integration as a future enhancement, which softens rather than fully contradicts the limitation, but it still creates intent ambiguity about whether sentiment is in scope for this skill's profiling purpose.

Content

No source excerpt is available for this finding.

Unpinned Dependencies

Low
Category
Supply Chain
Confidence
95% confidence
Finding

The dependency is specified with only a minimum version, so installs can drift to different releases over time and may pull in versions with newly introduced flaws or breaking behavior. In a security-sensitive workflow, unpinned dependencies reduce reproducibility and make it harder to verify whether a known-vulnerable or fixed version is being installed.

Content

Scanner excerpt Β· requirements-test.txt (reported line 5)May include surrounding context.

text
# Install with: pip install -r requirements-test.txt

# Core skill requirements (from requirements.txt)
scikit-learn>=1.3.0
numpy>=1.24.0
PyYAML>=6.0

Unverifiable Dependency: scikit-learn has 6 known advisory(ies) (CVE-2020-13092 (scikit-learn Deserialization of Untrusted Data); CVE-2024-5206 (scikit-learn sensitive data leakage vulnerability); CVE-2020-28975 (scikit-learn Denial of Service) +3 more), but the manifest does not pin a version, so it is unknown whether the installed release is affected

Low
Category
Supply Chain
Confidence
88% confidence
Finding

Because scikit-learn is not pinned, the manifest does not prove whether deployment will use a version affected by known advisories or a patched one. This creates unverifiable exposure and complicates risk assessment, even though the file alone does not confirm exploitation.

Content

No source excerpt is available for this finding.

Unpinned Dependencies

Low
Category
Supply Chain
Confidence
95% confidence
Finding

Using an unpinned numpy version means the resolved package can vary by environment and time, which weakens build reproducibility and can unintentionally introduce vulnerable or incompatible releases. This is especially relevant where transitive dependency resolution may change without review.

Content

Scanner excerpt Β· requirements-test.txt (reported line 6)May include surrounding context.

text
# Core skill requirements (from requirements.txt)
scikit-learn>=1.3.0
numpy>=1.24.0
PyYAML>=6.0

# WildChat dataset access

Unverifiable Dependency: numpy has 16 known advisory(ies) (CVE-2014-1859 (Numpy arbitrary file write via symlink attack); CVE-2021-41495 (NumPy NULL Pointer Dereference); CVE-2021-33430 (NumPy Buffer Overflow (Disputed)) +13 more), but the manifest does not pin a version, so it is unknown whether the installed release is affected

Low
Category
Supply Chain
Confidence
86% confidence
Finding

An unpinned numpy requirement prevents reviewers from determining whether the installed version is affected by any of the known advisories listed by the scanner. The main risk is supply-chain uncertainty rather than a guaranteed exploitable flaw in this file by itself.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.