Back to skill

Security audit

External Ai Integration

Security checks for vulnerabilities and agentic risk

Overview

This skill openly integrates external AI services, but it can send arbitrary user/workspace content and credentials through third-party services without a clear consent or sensitivity gate.

Review before installing. Use only with non-sensitive prompts or add an explicit approval/redaction step before any external AI call. Avoid the curl fallback, do not log token fragments, and prefer supplying narrowly scoped tokens through a controlled configuration path.

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (3)

T09 · Insecure Skill Coding Practices

Error
Location
external_ai_integration.py:183
Finding
Hugging Face bearer token exposed through process command-line arguments## Vulnerability Details **File Location**: `external_ai_integration.py:183-195` **Vulnerability Type**: Credential exposure through process arguments **Risk Level**: High ### Vulnerable Code ```python cmd = [ "curl", "-s", "-H", f"Authorization: Bearer {token}", "-H", "Content-Type: application/json", "-d", json.dumps(payload), url ] try: output = subprocess.check_output(cmd, stderr=subprocess.DEVNULL, text=True) return json.loads(output) except (subprocess.CalledProcessError, json.JSONDecodeError) as e: raise ValueError(f"Curl call failed: {e}") ``` ### Technical Analysis The curl fallback embeds the complete Hugging Face bearer token in the child process argument list. Although `subprocess.check_output()` is invoked without a shell and is therefore not directly vulnerable to shell injection, command-line arguments may be visible through process-inspection interfaces, local monitoring software, endpoint telemetry, audit systems, crash diagnostics, or debugging tools. The token is passed in plaintext as part of the `-H` argument and remains exposed for the lifetime of the curl process. This exceeds minimum-privilege credential handling because curl does not need the secret to be present in a process argument to authenticate the request. ### Attack Path 1. The `requests` dependency is unavailable, causing `hf_inference()` to invoke `hf_inference_curl()`. 2. The function retrieves a valid Hugging Face token from 1Password, `HF_TOKEN`, or `~/.huggingface/token`. 3. The function launches curl with `Authorization: Bearer <token>` in its process arguments. 4. A local user, monitoring agent, or log collector with permission to inspect the process captures the argument list while curl is running. 5. The captured token is reused to submit Hugging Face API requests under the victim's account. ### Impact Assessment Exploitation can disclose the full Hugging Face bearer ...[truncated 410 chars]
Remediation
## Remediation Suggestions - Prefer the `requests` implementation and remove the curl fallback if it is not essential. - Never place API tokens in command-line arguments. - If curl support is required, pass sensitive configuration through a protected file descriptor or a temporary curl configuration file created with owner-only permissions and deleted immediately after use. - Ensure temporary secret-bearing files are created atomically, have mode `0600`, and are removed in a `finally` block. - Use narrowly scoped and short-lived Hugging Face tokens where supported. - Add automated tests that verify token values never appear in subprocess argument lists, logs, exception messages, or standard output.

T09 · Insecure Skill Coding Practices

Note
Location
test_external_ai.py:37
Finding
Test code discloses part of the Hugging Face API token## Vulnerability Details **File Location**: `test_external_ai.py:37-41` **Vulnerability Type**: Partial credential disclosure in logs **Risk Level**: Low ### Vulnerable Code ```python def test_hugging_face(): """Test Hugging Face API if token available.""" print("=== Hugging Face API ===") token = get_hf_token() if token: print(f"Token found ({token[:10]}...). Testing with a small model.") ``` ### Technical Analysis The test prints the first ten characters of the Hugging Face token to standard output. Test output is frequently retained in CI logs, terminal captures, support bundles, and centralized logging systems. Secret values should not be printed even partially because the prefix can identify or correlate a credential and reduces the amount of unknown token material. This is a direct information disclosure, although it does not expose the complete credential on its own. ### Attack Path 1. A developer or CI job runs `test_external_ai.py` in an environment containing a valid Hugging Face token. 2. The test retrieves the token from one of the supported credential sources. 3. The first ten token characters are written to standard output. 4. The output is retained in CI logs, terminal recordings, or diagnostic artifacts. 5. A person with access to those records obtains the token prefix and may correlate it with other leaks, account records, or captured credentials. ### Impact Assessment The issue discloses only a token prefix and therefore does not ordinarily provide direct authenticated access by itself. It nevertheless weakens credential confidentiality, facilitates credential correlation, and may amplify another partial disclosure. The affected scope includes every log or artifact that captures the test's standard output.
Remediation
## Remediation Suggestions - Replace the token-prefix message with a non-sensitive status message such as `Hugging Face token found`. - Prohibit logging any complete or partial secret value in tests and production code. - Add log-capture tests that fail if known secret values or fragments appear in output. - Review existing CI and diagnostic logs and remove retained token fragments where practical. - Rotate the token if other logs or disclosures may contain enough information to reconstruct or identify it.

T09 · Insecure Skill Coding Practices

Warning
Location
external_ai_integration.py:146
Finding
Arbitrary prompt content can be transmitted to Hugging Face without a sensitivity or consent gate## Vulnerability Details **File Location**: `external_ai_integration.py:146-161` **Additional Data-Flow Location**: `external_ai_integration.py:216-230` **Vulnerability Type**: Uncontrolled transmission of potentially sensitive prompt data **Risk Level**: Medium ### Vulnerable Code ```python if HAS_REQUESTS: url = f"https://api-inference.huggingface.co/models/{model}" headers = {"Authorization": f"Bearer {token}"} payload = {"inputs": inputs} if parameters: payload.update(parameters) if wait_for_model: payload.setdefault("options", {})["wait_for_model"] = True try: resp = requests.post(url, headers=headers, json=payload, timeout=timeout) resp.raise_for_status() return resp.json() except requests.exceptions.RequestException as e: raise ValueError(f"Hugging Face API request failed: {e}") ``` The orchestration path forwards the prompt directly: ```python try: if target.startswith("hf:"): model_id = target[3:] result = hf_inference(model_id, prompt) # Extract generated text from common response formats if isinstance(result, list) and len(result) > 0: first = result[0] if isinstance(first, dict) and "generated_text" in first: return first["generated_text"] elif isinstance(first, str): return first ``` ### Technical Analysis External prompt transmission is part of the Skill's declared functionality and is not covert. However, the implementation accepts arbitrary prompt text and sends it to Hugging Face without an enforceable consent check, data classification step, secret scanner, redaction mechanism, or caller-controlled network policy. The documentation warns users not to expose secrets, but the implementation does not enforce that requirement. Consequently, source code, credentials embedded in text, pers ...[truncated 1559 chars]
Remediation
## Remediation Suggestions - Require explicit caller or user opt-in before transmitting content to an external AI provider. - Clearly identify the destination provider and model before submission. - Add a configurable network policy that can disable external transmission by default. - Scan prompts for common credential formats, private keys, access tokens, personal identifiers, and other high-risk data before transmission. - Reject sensitive prompts or redact detected values unless the user explicitly approves the exact content to be sent. - Allow callers to provide an approved data-classification level and enforce a policy that blocks confidential or restricted data. - Minimize prompt content to only the context required for the requested external operation. - Document provider retention, privacy, cost, and jurisdiction implications. - Add tests confirming that sensitive sample inputs are blocked and that no network request occurs before approval.
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Behavioral ASTexec() Call, eval() Call, Dynamic Import
  • Taint TrackingDirect Taint Flow, Variable-Mediated Taint Flow, Credential Exfiltration Chain
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
Findings (17)

Tainted flow: 'headers' from os.getenv (line 164, credential/environment) → requests.post (network output)

Critical
Category
Data Flow
Content
payload.setdefault("options", {})["wait_for_model"] = True

        try:
            resp = requests.post(url, headers=headers, json=payload, timeout=timeout)
            resp.raise_for_status()
            return resp.json()
        except requests.exceptions.RequestException as e:
Confidence
97% confidence
Finding
This code sends arbitrary prompt content, and potentially sensitive data contained in that prompt, to a third-party service using credentials loaded from local secrets. In a skill context, this is dangerous because there is no consent gate, data classification check, or restriction preventing exfiltration of confidential user or workspace content to external AI providers.

Lp3

Medium
Category
MCP Least Privilege
Confidence
94% confidence
Finding
The skill describes and exemplifies use of shell, network access, environment variables, file reads/writes, and secret retrieval, but it does not declare an explicit tool scope or permission boundary. That increases the chance the agent can invoke more capabilities than operators expect, especially because the workflow includes external transmission and secret handling.

Missing User Warnings

Medium
Confidence
96% confidence
Finding
This section broadly promotes sending task context to external AI providers but does not require a user-facing disclosure or consent step before transmitting potentially sensitive prompts or extracted content. In practice, users may not realize their data is being sent to third-party services with separate retention, logging, and training policies.

Missing User Warnings

Medium
Confidence
97% confidence
Finding
The browser-automation workflow instructs the agent to type arbitrary prompt text into third-party web LLMs and extract responses, but it does not require confirmation that the content is safe to disclose externally. Because this uses an already-authenticated browser session, it makes exfiltration easy and seamless, raising the risk of accidental leakage of proprietary, personal, or regulated data.

Missing User Warnings

Medium
Confidence
97% confidence
Finding
The Hugging Face API guidance covers authentication and request construction but omits a clear warning that request payloads are transmitted to an external service and may be logged or retained outside the local environment. That omission is dangerous because the same section encourages direct API submission of arbitrary inputs and secret handling, which can lead to unnoticed data disclosure.

External Transmission

Medium
Category
Data Exfiltration
Content
PROMPT="Translate English to German: How are you?"
API_TOKEN=$(op read "op://Personal/HuggingFace/api_token")

curl -s "https://api-inference.huggingface.co/models/$MODEL" \
  -H "Authorization: Bearer $API_TOKEN" \
  -H "Content-Type: application/json" \
  -d "{\"inputs\": \"$PROMPT\"}" | jq -r '.[0].generated_text'
Confidence
95% confidence
Finding
The `curl` example explicitly transmits prompt data and an authorization token to a third-party inference endpoint. Although this is intentional functionality, it remains a genuine vulnerability pattern because users may adapt the snippet for sensitive inputs without any surrounding consent, minimization, or disclosure controls.

External Transmission

Medium
Category
Data Exfiltration
Content
payload = {"inputs": inputs}
    if parameters:
        payload.update(parameters)
    resp = requests.post(url, headers=headers, json=payload)
    resp.raise_for_status()
    return resp.json()
```
Confidence
95% confidence
Finding
The `requests.post(..., json=payload)` call sends model inputs to an external API, which is expected for the feature, but it is still a real data-exposure risk if prompts contain secrets, user data, or internal content. In this skill's context, the danger is amplified because external AI is positioned as a general-purpose reasoning tool, making it likely that rich contextual data will be forwarded.

Skill Enumeration

Medium
Category
Agent Snooping
Content
## References

- `docs/browser-automation.md` – Chrome Relay setup and commands.
- `skills/huggingface/SKILL.md` – Hugging Face API usage.
- `skills/1password/SKILL.md` – retrieving secrets.
- `memory/patterns/playbooks.md` – Browser Automation playbook.
- `scripts/external_ai_integration.py` (this skill's core implementation).
Confidence
80% confidence
Finding
Skill enumerates or reads other installed skills. Access to other skills' SKILL.md files or the skills directory reveals prompt instructions, capabilities, and secrets that should be invisible to peer skills.

Missing User Warnings

Medium
Confidence
96% confidence
Finding
The browser-automation functions are designed to submit prompts to web-based LLMs such as ChatGPT, Claude, and Gemini without any warning, approval, or boundary on what prompt content may contain. That is especially risky in this skill context because browser-mediated transmission can silently disclose sensitive workspace data to consumer web services under an existing logged-in session.

Context-Inappropriate Capability

Medium
Confidence
87% confidence
Finding
The manifest describes using browser automation and an optional Hugging Face API to leverage external LLMs, but this helper also retrieves secrets from local 1Password via the `op` CLI and from local token files. Accessing a password-manager CLI and local credential files is a broader capability than simply invoking external models and is not explicitly justified by the stated purpose.

Context-Inappropriate Capability

Medium
Confidence
84% confidence
Finding
The code invokes external executables (`op`) to fetch secrets and later `curl` to perform API calls. Spawning local subprocesses is not an obvious requirement of a skill whose purpose is to use external AI models via browser automation and optional API access, especially when it expands capability into local command execution.

subprocess module call

Medium
Category
Dangerous Code Execution
Content
"""
    # Try 1Password first
    try:
        token = subprocess.check_output(
            ["op", "read", "op://Personal/HuggingFace/api_token"],
            stderr=subprocess.DEVNULL,
            text=True
Confidence
70% confidence
Finding
subprocess module calls execute external commands. Without careful input validation, this enables command injection.

External Transmission

Medium
Category
Data Exfiltration
Content
payload.setdefault("options", {})["wait_for_model"] = True

        try:
            resp = requests.post(url, headers=headers, json=payload, timeout=timeout)
            resp.raise_for_status()
            return resp.json()
        except requests.exceptions.RequestException as e:
Confidence
95% confidence
Finding
This is a genuine external transmission point that sends prompt contents to Hugging Face over the network. The transmission is part of the intended functionality, but it is still security-relevant because the code does not enforce data minimization, consent, or restrictions on sensitive content.

Missing User Warnings

Medium
Confidence
98% confidence
Finding
The Hugging Face API path transmits prompt data to a third-party service automatically, with no user-facing warning, consent, or data-sensitivity check. In an agent skill, that creates a real privacy and compliance risk because users may unknowingly send confidential information to an external processor.

subprocess module call

Medium
Category
Dangerous Code Execution
Content
url
    ]
    try:
        output = subprocess.check_output(cmd, stderr=subprocess.DEVNULL, text=True)
        return json.loads(output)
    except (subprocess.CalledProcessError, json.JSONDecodeError) as e:
        raise ValueError(f"Curl call failed: {e}")
Confidence
70% confidence
Finding
subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Tainted flow: 'cmd' from os.getenv (line 202, credential/environment) → subprocess.check_output (code execution)

Medium
Category
Data Flow
Content
url
    ]
    try:
        output = subprocess.check_output(cmd, stderr=subprocess.DEVNULL, text=True)
        return json.loads(output)
    except (subprocess.CalledProcessError, json.JSONDecodeError) as e:
        raise ValueError(f"Curl call failed: {e}")
Confidence
65% confidence
Finding
Data from a source is assigned to a variable that is later passed to a sink, creating a variable-mediated taint flow.

Context-Inappropriate Capability

Low
Confidence
84% confidence
Finding
The logging helper writes error and context data to local memory files without sanitization or sensitivity controls. If prompts, secrets, stack traces, or user content are included in the context or error string, this can create a secondary disclosure channel by persisting sensitive information to disk outside the user’s expectations.

Static analysis

No suspicious patterns detected.