Back to skill

Security audit

Family Soul Analyzer

Security checks for vulnerabilities and agentic risk

Overview

The skill has a coherent persona-generation purpose, but it mishandles private chat data, embedded credentials, and generated agent prompts enough that users should review it carefully before installing.

Treat this as requiring careful review before installation. Do not run it on real chat exports unless bundled private data and hardcoded credentials are removed, input processing is limited to the chosen file, and explicit consent, redaction, provider disclosure, and review-before-install controls are added for generated agent prompts.

Vulnerability Patterns
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Insecure DependenciesIntroduces malicious components through unsafe dependency sources
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Findings (7)

T09 · Insecure Skill Coding Practices

Error
Location
pipeline/03_extract_simple.py:16
Finding

Hard-Coded Kimi API Credential in Executable Pipeline Scripts

Content
View full analysis
Remediation
View remediation

T09 · Insecure Skill Coding Practices

Error
Location
pipeline/01_parse.py:184
Finding

Requested Input File Is Ignored and Bundled Private Chat Data Is Processed

Content
View full analysis
Remediation
View remediation

T09 · Insecure Skill Coding Practices

Error
Location
pipeline/03_extract.py:36
Finding

Unredacted Private Chat Content Is Transmitted to External LLM Providers

Content
View full analysis
str: lines = [] for m in msgs: role = m.get("role", "unknown") display_map = { "dad": "Father", "mom": "Mother", "son": "Child", "child": "Child", "unknown": m["sender"], } speaker = display_map.get(role, m["sender"]) lines.append(f"[{m['timestamp'][:16]}] {speaker}: {m['content']}") return "\n".join(lines) ``` The resulting text is inserted directly into an API prompt: ```python def build_request(chunk: dict, prompt_template: str) -> dict: text = chunk["text"] chunk_id = chunk["chunk_id"] time_range = f"{chunk['start_time']} ~ {chunk['end_time']}" prompt = ( prompt_template .replace("{{CHAT_TEXT}}", text) .replace("{{CHUNK_ID}}", str(chunk_id)) .replace("{{TIME_RANGE}}", time_range) ) return { "custom_id": f"chunk-{chunk_id}", "params": { "model": EXTRACT_MODEL, "max_tokens": 4096, "messages": [{"role": "user", "content": prompt}], }, } ``` Requests are then submitted externally: ```python batch = client.messages.batches.create(requests=batch_requests) ``` Alternate scripts transmit content to `https://api.kimi.com/coding/` or `https://api.moonshot.cn/v1`. ### Technical Analysis The denoising stage removes short messages, media placeholders, and duplicates, but it does not redact names, account identifiers, contact details, credentials, financial information, medical info ...[truncated 1150 chars]
Remediation
View remediation

T01 · Skill Instruction Hijacking

Error
Location
prompts/extract_patterns.md:5
Finding

Untrusted Chat Instructions Can Propagate into Installable Agent System Prompts

Content
View full analysis
{{CHAT_TEXT}} ``` The generated observations are then converted back into prompt text: ```python for i, obs in enumerate(high_signal): lines.append(f"\n--- Observation {i+1} ({obs.get('time_range', '')}) ---") o = obs.get("observations", {}) for key, label in [ ("emotional_patterns", "Emotional patterns"), ("values", "Values"), ("relational_dynamics", "Relationship dynamics"), ("collective_identity", "Collective identity"), ]: val = o.get(key, "") if val and val != "No clear signal": lines.append(f" {label}: {val}") ``` The second prompt explicitly targets a future system prompt: ```markdown This document will become the foundation of an AI agent's system prompt. ``` The persona prompt similarly embeds generated `soul.md` content: ```markdown ## Family collective persona (soul.md) {{SOUL_CONTENT}} ``` ### Technical Analysis XML-like delimiters provide formatting but do not establish an instruction hierarchy. The prompts do not explicitly state that chat text, quotations, observations, and prior generated documents are untrusted data that must never be interpreted as instructions. This creates a two-stage injection chain: 1. Untrusted chat content influences the extraction model. 2. Extraction output is trusted as observation data. 3. Observation data influences synthesis. 4. Synthesis output is intended to become a persistent Agent s ...[truncated 1265 chars]
Remediation
View remediation

other

Warning
Location
prompts/synthesize_persona.md:3
Finding

Persona Template Encourages Undisclosed Impersonation of Real People

Content
View full analysis
Remediation
View remediation

T09 · Insecure Skill Coding Practices

Warning
Location
pipeline/04_synthesize.py:34
Finding

Output Configuration Is Ignored and Stage 4 Imports an Undefined Setting

Content
View full analysis
Remediation
View remediation

T08 · Insecure Dependencies

Note
Location
requirements.txt:1
Finding

Unpinned and Incomplete Runtime Dependency Declaration

Content
View full analysis
=0.40.0 httpx>=0.25.0 ``` Executable Kimi scripts also import `openai`, but that package is not declared: ```python from openai import OpenAI ``` ### Technical Analysis Open-ended minimum constraints allow future versions to be installed without review. This makes deployments non-reproducible and can introduce incompatible behavior or newly compromised dependency releases. The undeclared `openai` dependency means alternate scripts either fail or use whichever globally installed module happens to resolve under that name. The audit did not identify a typosquatted package or explicitly malicious dependency source, so the finding concerns unsafe dependency management rather than a confirmed malicious package. ### Attack Path 1. A user runs `pip install -r requirements.txt`. 2. The package index resolves versions newer than those tested by the author. 3. A compromised, vulnerable, or incompatible release is installed. 4. Alternatively, an undeclared `openai` module already present in the environment is imported by an alternate script. 5. Dependency code executes with the privileges of the user running the Skill. ### Impact Assessment The maximum impact is arbitrary code execution under the installing or executing user's account if a resolved dependency is compromised. More likely impacts include runtime failure, incompatible API behavior, and inconsistent security properties across installations. ]]>
Remediation
View remediation
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
Findings (67)

Missing User Warnings

High
Category
Not specified by scanner
Confidence
98% confidence
Finding

Requiring an ANTHROPIC_API_KEY strongly indicates that chat contents may be sent to a third-party AI provider, yet the README does not clearly disclose that family chat exports could leave the local environment. This omission is dangerous because users may submit private conversations involving multiple people without informed consent or understanding of external processing, retention, and compliance implications.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
96% confidence
Finding

Even within SKILL.md alone, the privacy-critical behavior of uploading family group-chat contents to an external LLM provider is not surfaced in the top-level description or trigger/consent language. This is especially dangerous because the data category is highly sensitive interpersonal communication, and users may not expect third-party processing from a skill advertised primarily as persona extraction.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding

Even within SKILL.md alone, the privacy-critical behavior of uploading family group-chat contents to an external LLM provider is not surfaced in the top-level description or trigger/consent language. This is especially dangerous because the data category is highly sensitive interpersonal communication, and users may not expect third-party processing from a skill advertised primarily as persona extraction.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding

Even within SKILL.md alone, the privacy-critical behavior of uploading family group-chat contents to an external LLM provider is not surfaced in the top-level description or trigger/consent language. This is especially dangerous because the data category is highly sensitive interpersonal communication, and users may not expect third-party processing from a skill advertised primarily as persona extraction.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding

Even within SKILL.md alone, the privacy-critical behavior of uploading family group-chat contents to an external LLM provider is not surfaced in the top-level description or trigger/consent language. This is especially dangerous because the data category is highly sensitive interpersonal communication, and users may not expect third-party processing from a skill advertised primarily as persona extraction.

Content

No source excerpt is available for this finding.

Missing User Warnings

High
Category
Not specified by scanner
Confidence
99% confidence
Finding

The skill processes highly sensitive family chat logs and describes use of external LLM APIs, yet it does not present a clear warning or consent notice about privacy, third-party transfer, retention, or the risks of analyzing intimate communications. Lack of explicit warning undermines informed consent and can expose users and non-consenting family members to serious privacy harm.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

High
Category
Not specified by scanner
Confidence
99% confidence
Finding

The file stores directly identifying group metadata including chatroom ID, group name, message volume, timestamps, and remote avatar URLs, while the skill's purpose is to derive persistent personas from family chat logs. This creates a strong re-identification risk and enables profiling of real individuals and their social graph without any visible consent, minimization, or privacy boundary in the artifact.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

High
Category
Not specified by scanner
Confidence
97% confidence
Finding

The dataset contains identifiable individuals' whereabouts, travel plans, city-level locations, routes, and timing details embedded in natural-language chat. In a skill designed to synthesize 'digital personas,' this materially increases stalking, doxxing, and behavioral profiling risk because location traces can be linked to named family members over time.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

High
Category
Not specified by scanner
Confidence
99% confidence
Finding

A hard-coded fallback API key is embedded directly in source code, allowing anyone with code access to use the credential and enabling outbound processing of sensitive chat data without meaningful user control. This is especially dangerous here because it combines credential exposure with automated transmission of private family conversations to an external service.

Content

No source excerpt is available for this finding.

Missing User Warnings

High
Category
Not specified by scanner
Confidence
98% confidence
Finding

The fallback API key makes it possible for the skill to perform external API calls even when the user has not intentionally configured remote access, undermining informed consent and security expectations. In the context of persona extraction from family chats, this materially increases the chance of undisclosed transfer of highly sensitive personal data.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

High
Category
Not specified by scanner
Confidence
99% confidence
Finding

This skill processes highly sensitive family chat logs, then configures a remote third-party API client and sends prompt content derived from those chats off-device. That exceeds a reasonable expectation for a persona-extraction tool unless it is explicitly disclosed and consented to, because intimate conversations may contain personal, financial, health, or relationship data.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

High
Category
Not specified by scanner
Confidence
100% confidence
Finding

A hardcoded API key is embedded directly in the source code, which exposes the credential to anyone with repository or package access and enables unauthorized use of the vendor account. In a skill handling family chat data, this is especially dangerous because it combines sensitive data export capability with a reusable secret that cannot be scoped safely once leaked.

Content

No source excerpt is available for this finding.

Missing User Warnings

High
Category
Not specified by scanner
Confidence
99% confidence
Finding

The code includes and uses a hardcoded external API credential without any visible user-facing disclosure, which hides a meaningful remote capability from users. This undermines informed consent and creates supply-chain risk, since the bundled key could route usage and sensitive chat processing through an account controlled by the skill author or another party.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

High
Category
Not specified by scanner
Confidence
99% confidence
Finding

A hardcoded API key is embedded directly in source code, which exposes the credential to anyone with repository or package access and enables unauthorized use of the external service. In a skill handling private family chat data, leaked credentials can also facilitate uncontrolled data submission and billing abuse.

Content

No source excerpt is available for this finding.

Missing User Warnings

High
Category
Not specified by scanner
Confidence
99% confidence
Finding

The sensitive credential is stored in plaintext in code without safer handling or user disclosure, making accidental disclosure extremely likely through source sharing, logs, backups, or package publication. This is a straightforward secret-management failure with direct abuse potential.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

High
Category
Not specified by scanner
Confidence
96% confidence
Finding

The code transmits family group chat content to a third-party LLM service via client.messages.create, including up to 3000 characters of each chat chunk. Because the skill processes highly sensitive interpersonal conversations and the external transfer is not transparently disclosed in the described skill behavior, this creates a serious privacy and data-handling risk.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

High
Category
Not specified by scanner
Confidence
99% confidence
Finding

The file contains a hard-coded API key and fixed remote endpoint, which exposes a secret directly in source code and enables unauthorized reuse if the repository, package, or logs are shared. In a tool handling intimate family-chat analysis, embedding external access credentials also increases the likelihood of silent data exfiltration through a preconfigured remote service.

Content

No source excerpt is available for this finding.

Missing User Warnings

High
Category
Not specified by scanner
Confidence
97% confidence
Finding

The script transmits sensitive chat-derived observations to an external API without any user-facing warning in this file. Because the content concerns family group chats and personality synthesis, the data can include deeply personal behavioral inferences, making undisclosed transmission especially privacy-invasive.

Content

No source excerpt is available for this finding.

Missing User Warnings

High
Category
Not specified by scanner
Confidence
96% confidence
Finding

This prompt explicitly processes real family chat logs to derive a digital family profile, which is highly privacy-sensitive data involving multiple people, likely including third parties who did not consent. The absence of any minimization, consent, redaction, retention, or warning language increases the risk of unauthorized profiling, overcollection, and downstream misuse of intimate household information.

Content

No source excerpt is available for this finding.

Ssd 3

High
Category
Not specified by scanner
Confidence
98% confidence
Finding

The prompt directs the model to infer emotional patterns, values, relationship dynamics, and collective identity from real family conversations, creating a sensitive personality dossier. In the context of an AI-agent persona substrate, this can enable invasive profiling, manipulation, deanonymization, or persistent storage of deeply personal inferences that go beyond the original conversational purpose.

Content

No source excerpt is available for this finding.

Env Variable Harvesting

High
Category
Data Exfiltration
Confidence
84% confidence
Finding

The runner copies the full parent environment into child pipeline stages, which can expose unrelated secrets such as API keys, tokens, proxy credentials, or cloud auth material to every invoked stage. In this skill context, the pipeline processes sensitive family chat data and may include LLM-facing extraction stages, so broad environment propagation increases the blast radius if any downstream stage logs, transmits, or mishandles inherited secrets.

Content

Scanner excerpt · scripts/run_forge.py (reported line 69)May include surrounding context.

python
emit(f"STAGE:{stage_num}:START", label)

    env = os.environ.copy()
    env["SOUL_FORGE_SOURCE"] = str(source_file)
    env["SOUL_FORGE_OUTPUT"] = str(OUTPUT_DIR)
    env["PYTHONPATH"] = str(PIPELINE_DIR)

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

The README promotes converting family group chat exports into durable persona artifacts without any visible privacy warning, consent guidance, retention policy, or handling precautions. Because the input is highly sensitive family communications and the outputs are structured personality profiles, users may unknowingly expose intimate data and create secondary sensitive data that can be misused or retained indefinitely.

Content

No source excerpt is available for this finding.

Undeclared Tool Scope

Medium
Category
MCP Least Privilege
Confidence
94% confidence
Finding

The skill requests and describes capabilities equivalent to shell execution, filesystem access, environment-variable access, and outbound API use, but does not declare any explicit tool scope or permission boundaries. In a skill that processes private family chat logs, missing scope declarations increases the chance of over-privileged execution and makes user review of data access and exfiltration risk much harder.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

The trigger phrases are broad enough to overlap with ordinary requests such as 'analyze chat records' or 'generate persona,' which can cause accidental invocation of a skill that runs shell commands and uploads sensitive chat data. In this context, mis-triggering is more dangerous than usual because the underlying workflow handles private family conversations and external API calls.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
96% confidence
Finding

The chat includes sensitive health-adjacent and body-related information such as weight, body fat, sleep, pain, and medical concerns tied to identifiable participants. Because the skill's stated goal is to distill stable personal traits, these details can be overfit into long-lived profiles and misused for discrimination, embarrassment, or intrusive inference about health status.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.