Back to skill

Security audit

Voight-Kampff Test

Security checks for vulnerabilities and agentic risk

Overview

This skill is a themed empathy test, but it asks for sensitive personal observations and may save assessment reports without clear consent, scope, or retention controls.

Install only if you intend to run a Chinese-language Blade Runner-style assessment in a controlled, clearly opt-in context. Do not use it for real identity, employment, security, legal, medical, or mental-health judgments. Before use, tell subjects what will be asked and recorded, avoid collecting facial or pupil observations, allow skipping questions, and disable or tightly control report saving.

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (1)

T09 · Insecure Skill Coding Practices

Warning
Location
scripts/test_runner.py:288
Finding

Fixed Output Path Permits Symlink-Based File Overwrite

Content
View full analysis

Vulnerability Details

File Location: scripts/test_runner.py, lines 288-291
Vulnerability Type: CWE-59 — Improper Link Resolution Before File Access
Risk Level: Medium

Vulnerable Code

python
# Save to JSON
result_dict = asdict(result)
with open("/home/gem/.aily/workspace/skills/voight-kampff-test/results/demo_result.json", "w") as f:
    json.dump(result_dict, f, indent=2, ensure_ascii=False)

Technical Analysis

The demonstration routine writes its result to a fixed, absolute path using open(..., "w"). This mode follows symbolic links and truncates the resolved destination before writing. The code does not verify that the destination is a regular file, validate ownership, prevent symbolic-link traversal, or create the output securely.

If another local user can control the results directory or replace demo_result.json with a symbolic link, running the script under a more privileged account can overwrite a file accessible to that account. The written content is constrained to the generated JSON report, so this is not an arbitrary-content write; nevertheless, truncation and replacement of the target's contents may cause data loss or service disruption.

The hardcoded environment-specific path can also cause execution failure when the directory does not exist, although that reliability problem is secondary to the unsafe file-handling issue.

Attack Path

  1. An attacker obtains write access to /home/gem/.aily/workspace/skills/voight-kampff-test/results/ or otherwise controls demo_result.json.

  2. The attacker creates a symbolic link from demo_result.json to a file writable by the account expected to run the skill:

    bash
    ln -s /path/to/target /home/gem/.aily/workspace/skills/voight-kampff-test/results/demo_result.json
    
  3. A more privileged user or agent runs:

    bash
    python scripts/test_runner.py
    
  4. Python follows the symbolic link ...[truncated 922 chars]

Remediation
View remediation

Remediation Suggestions

  • Do not write demonstration output to a hardcoded absolute path. Accept an explicit output path from the caller or use a directory owned exclusively by the current user.
  • Create the destination directory with restrictive permissions and verify its ownership before writing.
  • Refuse to follow symbolic links. On supported platforms, open the file using os.open() with O_NOFOLLOW, O_CREAT, and O_EXCL, then wrap the descriptor with os.fdopen().
  • Validate the opened file with os.fstat() and ensure that it is a regular file owned by the expected user.
  • Prefer atomic output: securely create a temporary file in the same trusted directory, flush and synchronize it, and then use os.replace() after validating the destination.
  • Apply restrictive file permissions, such as 0o600, if reports may contain subject responses or other sensitive information.
  • For a demonstration script that does not need persistence, print the report only or use Python's tempfile module rather than a predictable shared path.

A hardened approach should resemble:

python
import os
from pathlib import Path

output_dir = Path.home() / ".local" / "share" / "voight-kampff-test"
output_dir.mkdir(mode=0o700, parents=True, exist_ok=True)
output_path = output_dir / "demo_result.json"

flags = os.O_WRONLY | os.O_CREAT | os.O_EXCL
if hasattr(os, "O_NOFOLLOW"):
    flags |= os.O_NOFOLLOW

fd = os.open(output_path, flags, 0o600)
with os.fdopen(fd, "w", encoding="utf-8") as f:
    json.dump(result_dict, f, indent=2, ensure_ascii=False)

If overwriting an existing report is required, use a securely created temporary file and an atomic replacement strategy rather than opening a predictable destination directly.

Vulnerability Patterns
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
Findings (16)

Missing User Warnings

High
Category
Not specified by scanner
Confidence
97% confidence
Finding

The skill instructs the operator to record verbatim responses, reaction time, and even biometric-style or health-adjacent observations such as pupil changes and facial flushing, but provides no warning, consent flow, or limitation on collecting this sensitive data. In context, this is especially risky because the skill frames these observations as evidence for classifying someone as 'human' or 'replicant,' encouraging invasive and potentially discriminatory profiling.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
83% confidence
Finding

The README is written entirely in Chinese and presents the skill behavior, scoring labels, and usage flow without offering any language choice, which can force a locale assumption onto users. In security terms this is lower severity than code execution issues, but it can still cause misunderstandings about consent, classification, or how the test operates, especially for multilingual deployments.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

The activation phrases are broad enough to match ordinary conversation such as '测试我' or '测测我是不是机器人', which can cause the skill to trigger outside clearly intended contexts. Because the skill performs identity-style classification and steers conversation into a test flow, accidental invocation could disrupt unrelated interactions and apply anthropomorphic or exclusionary judgments to users without explicit consent.

Content

No source excerpt is available for this finding.

Undeclared Tool Scope

Medium
Category
MCP Least Privilege
Confidence
78% confidence
Finding

The skill references an automation script and the static analyzer detected file-write capability, but the manifest declares no explicit tool scope or permissions. That mismatch is dangerous because an agent or runtime may invoke capabilities without clear user/admin visibility, increasing the risk of unauthorized report creation or persistence.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
72% confidence
Finding

The skill content and label are presented entirely in Chinese, and the instructions do not offer any language choice or indicate that Chinese is optional. This can violate language or locale policy when a skill implicitly forces a specific language without user opt-in or documented justification.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The activation scope is broad and ambiguous, including generic phrases like 'test me' and scenarios such as suspected AI-like behavior or agent self-evaluation. This can cause the skill to trigger without clear user intent and lead to sensitive profiling or judgmental classification of users or agents in inappropriate contexts.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The statement that all reports will be archived indicates retention of potentially sensitive behavioral assessment records without any warning about storage, access, retention period, or user choice. This creates privacy and misuse risk because classification results and raw responses may persist beyond the immediate interaction and be reused or exposed later.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
86% confidence
Finding

This markdown file is written entirely in Chinese and provides operational guidance in that language only. Under the policy, forcing a specific language without user opt-in is a natural-language policy violation unless the locale constraint is clearly documented and justified, which is not present here.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Medium
Category
Not specified by scanner
Confidence
92% confidence
Finding

L003 says the test is 'used to measure pupil contraction, capillary dilation' and similar signals, implying biometric or physiological measurement. In this file, however, the implementation artifact is only a static questionnaire with qualitative prompts and scoring dimensions, so the documentation overstates what the skill content actually supports.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
92% confidence
Finding

The skill presents multiple emotionally intense prompts about death, grief, trauma, fear, disability, betrayal, and moral dilemmas without advance warning, opt-in consent, or a safer alternative path. In an agent context, this can coerce users into disclosing sensitive psychological information, trigger distress, and create privacy and safety risks disproportionate to a casual 'test me' interaction.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The statement implying that reports will be archived suggests retention of highly sensitive emotional responses without informing users about storage, retention, access, or purpose. In this skill's context, the collected material could include grief, fear, family history, or other intimate disclosures, making undisclosed retention a meaningful privacy risk.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Medium
Category
Not specified by scanner
Confidence
92% confidence
Finding

The top-level documentation narrows the tool's purpose to AI-agent assessment. However, the actual questions and classification logic are written for general subjects and explicitly ask whether the subject is human, producing a HUMAN or REPLICANT label regardless of whether the subject is an AI agent.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The assessment prompts are presented exclusively in Chinese, which forces a specific language on users and subjects. Under the policy criteria, a fixed language/locale is a violation unless the skill offers user opt-in or clearly documents a justified region-specific constraint.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Low
Category
Not specified by scanner
Confidence
84% confidence
Finding

L179 states that 'all reports will be archived,' which describes data retention behavior. This file contains only question content and scoring notes, with no code or mechanism for storing, archiving, or handling reports, so the statement contradicts the observable behavior of this artifact.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

Low
Category
Not specified by scanner
Confidence
90% confidence
Finding

The demo persists a subject assessment containing identifier, timestamp, answers, and classification to a fixed filesystem path. Even though this is only in a demo path, the data is sensitive profiling output and is written without minimization, consent, retention controls, or access protection, which creates unnecessary privacy and information-disclosure risk.

Content

No source excerpt is available for this finding.

Missing User Warnings

Low
Category
Not specified by scanner
Confidence
92% confidence
Finding

The demo code silently writes a JSON report to disk without any warning in that execution path. Because the report includes sensitive responses and classification data, undisclosed persistence can surprise operators and leak personal or profiling information into local storage, logs, backups, or shared workspaces.

Content

No source excerpt is available for this finding.