Back to skill

Security audit

data-synthesis

Security checks for vulnerabilities and agentic risk

Overview

The skill largely does what it says, but API mode can send CSV text and an API bearer token to a configurable endpoint that may be plain HTTP.

Review before installing if you will process private, regulated, or proprietary CSVs. Use dry-run for local testing, prefer HTTPS endpoints, avoid sending API keys to HTTP URLs, and treat the generated JSONL as containing copies of source data because it stores chunks and source fields.

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (1)

T09 · Insecure Skill Coding Practices

Warning
Location
scripts/synthesize_qa.py:109
Finding

Cleartext Transmission of Corpus Data and API Credentials

Content
View full analysis

Vulnerability Details

File Location: scripts/synthesize_qa.py, lines 109-136
Vulnerability Type: Sensitive data and bearer credentials transmitted over an unrestricted HTTP endpoint
Risk Level: Medium

python
def complete(self, model: str, system: str, user: str) -> str:
    if self._base.endswith("/chat/completions"):
        url = self._base
    elif self._base.endswith("/v1"):
        url = f"{self._base}/chat/completions"
    else:
        url = f"{self._base}/v1/chat/completions"
    body: dict[str, Any] = {
        "model": model,
        "messages": [
            {"role": "system", "content": system},
            {"role": "user", "content": user},
        ],
        "max_tokens": int(self._cfg.max_tokens),
        "temperature": float(self._cfg.temperature),
    }
    data = json.dumps(body).encode("utf-8")
    headers = {"Content-Type": "application/json"}
    if self._cfg.api_key:
        headers["Authorization"] = f"Bearer {self._cfg.api_key}"
    req = urllib.request.Request(url, data=data, headers=headers, method="POST")
    try:
        with urllib.request.urlopen(req, timeout=self._cfg.timeout_sec) as resp:
            payload = json.loads(resp.read().decode("utf-8"))

Technical Analysis

The configurable OPENAI_BASE_URL is used without validating its URL scheme. Consequently, the client accepts http:// endpoints as well as HTTPS endpoints. The project documentation explicitly presents an HTTP endpoint as a configuration example.

In API mode, each request body contains CSV-derived corpus chunks in messages[].content. Answer-generation requests also contain generated questions. When OPENAI_API_KEY is configured, the same request carries the credential in a bearer Authorization header.

Over plain HTTP, neither the request body nor its headers receive transport confidentiality or integrity protection. A network-positioned attacker can therefore ob ...[truncated 1651 chars]

Remediation
View remediation

Remediation Suggestions

  1. Require https:// for all non-local API endpoints and reject cleartext URLs before constructing a request.
  2. If HTTP is needed for isolated development environments, require an explicit option such as --allow-insecure-http; do not enable it through the endpoint value alone.
  3. Restrict any HTTP exception to loopback or explicitly approved private endpoints and emit a prominent warning that request content is unencrypted.
  4. Never attach an Authorization header to an HTTP request. Fail closed if a credential is configured with an insecure endpoint.
  5. Update SKILL.md to use an HTTPS endpoint and clearly disclose that API mode sends source corpus text to the configured service.
  6. Consider endpoint allowlisting, certificate validation guidance for internal gateways, and preflight confirmation that displays the destination host before processing corpus data.
  7. Minimize transmitted data where feasible by redacting secrets or personally identifiable information before synthesis and by limiting chunk content to what the generation task requires.
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • System Prompt LeakageDirect Leakage, Indirect Extraction, Tool-Based Exfiltration
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
Findings (11)

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding

The declared description presents a full QA synthesis pipeline driven by an LLM and producing JSONL training data. The actual code chunk is a preparatory CSV inspection utility only. Its primary purpose is to validate corpus structure and preview contents, as also stated in its own docstring ('Phase 1 — preparation'). No network/API usage, model invocation, chunking, or dataset synthesis appears in this code. This is therefore a material description-behavior mismatch, not just an omitted implementation detail.

Content

No source excerpt is available for this finding.

Undeclared Tool Scope

Medium
Category
MCP Least Privilege
Confidence
90% confidence
Finding

The skill documents capabilities to read environment variables, write output files, and make network requests to an OpenAI-compatible endpoint, but it does not declare any explicit tool scope or permission boundary. This increases the risk of over-broad execution in agent environments because operators cannot easily see or constrain that the skill may exfiltrate corpus data to a remote service and write generated data to disk.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The skill instructs users to enable API mode and send corpus chunks to an OpenAI-compatible or internal model service, but it does not clearly warn that input data may leave the local environment. If users process sensitive CSV content, this can lead to unintended disclosure of proprietary or personal data to external or semi-external inference infrastructure.

Content

No source excerpt is available for this finding.

External Transmission

Medium
Category
Data Exfiltration
Confidence
60% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · scripts/synthesize_qa.py (reported line 75)May include surrounding context.

python
class LLMConfig:
    api_key: str | None = None
    # Can be either:
    # - Base URL ending with /v1 (or without), e.g. https://api.openai.com/v1
    # - Full chat completions URL, e.g. http://host/.../v1/chat/completions
    base_url: str = "https://api.openai.com/v1"
    model: str = "gpt-4o"

External Transmission

Medium
Category
Data Exfiltration
Confidence
60% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · scripts/synthesize_qa.py (reported line 77)May include surrounding context.

python
class LLMConfig:
    api_key: str | None = None
    # Can be either:
    # - Base URL ending with /v1 (or without), e.g. https://api.openai.com/v1
    # - Full chat completions URL, e.g. http://host/.../v1/chat/completions
    base_url: str = "https://api.openai.com/v1"
    model: str = "gpt-4o"

External Transmission

Medium
Category
Data Exfiltration
Confidence
60% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · scripts/synthesize_qa.py (reported line 146)May include surrounding context.

python
class LLMConfig:
    api_key: str | None = None
    # Can be either:
    # - Base URL ending with /v1 (or without), e.g. https://api.openai.com/v1
    # - Full chat completions URL, e.g. http://host/.../v1/chat/completions
    base_url: str = "https://api.openai.com/v1"
    model: str = "gpt-4o"

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

When API mode is enabled, the script sends chunked CSV text to a configurable external chat-completions endpoint without any consent gate, sensitivity check, or prominent warning at the transmission point. If operators run this on proprietary or regulated data, the skill can unintentionally exfiltrate corpus contents to third-party infrastructure.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
97% confidence
Finding

The system prompt hard-codes Chinese-language instructions for question generation, which steers outputs into a specific language regardless of the user's locale or the source text language. This is a natural-language locale policy issue because the file does not offer language selection or document an explicit regional constraint.

Content

No source excerpt is available for this finding.

Ssd 3

Medium
Category
Not specified by scanner
Confidence
96% confidence
Finding

The pipeline forwards raw chunk text to the LLM and then writes both the original chunk and auxiliary source fields verbatim into the output JSONL. This creates a compounded data-exposure path: sensitive input may leave the environment via the API call and then remain replicated in generated artifacts, increasing the blast radius for secrets, PII, or confidential business content.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
95% confidence
Finding

The manifest description is presented only in Chinese, which can impose a language constraint on users without opt-in. The policy allows locale-specific behavior when justified or optional, but this file does not provide an alternative language or explain that the skill is intentionally Chinese-only.

Content

No source excerpt is available for this finding.

Missing User Warnings

Low
Category
Not specified by scanner
Confidence
80% confidence
Finding

This code reads the selected text column from the input CSV and prints preview snippets to stdout in a JSON report. Although the module docstring mentions a text preview, there is no explicit runtime warning or caution that file contents may be displayed, which can expose sensitive data if the CSV contains private text.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.