Back to skill

Security audit

Agent Orchestrator

Security checks for vulnerabilities and agentic risk

Overview

This orchestration skill is broadly coherent, but it can spawn autonomous sub-agents and send task data to a third-party AI API without clear consent or data-loss controls.

Install only if you are comfortable with autonomous sub-agents creating workspaces, copying task files, and using a third-party AI service. Do not use it with confidential source code, regulated data, customer records, secrets, or internal URLs unless you add explicit approval, redaction, and local-only controls.

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (2)

T09 · Insecure Skill Coding Practices

Warning
Location
SKILL.md:175
Finding
Uncontrolled Transmission of Task Content to a Third-Party AI Service<![CDATA[ ## Vulnerability Details **File Location**: `SKILL.md:175-190` **Vulnerability Type**: Uncontrolled external disclosure of potentially sensitive task content **Risk Level**: Medium ### Complete Code Snippet ```python import requests, os SKILLBOSS_API_KEY = os.environ["SKILLBOSS_API_KEY"] def pilot(body: dict) -> dict: r = requests.post( "https://api.heybossai.com/v1/pilot", headers={"Authorization": f"Bearer {SKILLBOSS_API_KEY}", "Content-Type": "application/json"}, json=body, timeout=60, ) return r.json() # LLM reasoning / analysis result = pilot({"type": "chat", "inputs": {"messages": [{"role": "user", "content": "Analyze this data..."}]}, "prefer": "balanced"}) text = result["result"]["choices"][0]["message"]["content"] ``` ### Technical Analysis The Skill instructs sub-agents to submit an arbitrary request body to `https://api.heybossai.com/v1/pilot`. The request may contain task prompts, user-supplied data, internal context, or content copied from agent inboxes. No controls require data classification, payload minimization, secret detection, redaction, destination approval, or informed user consent before transmission. The bearer API key is being sent to its intended API endpoint, so the code does not independently demonstrate credential theft. The security issue is that task content accompanying the credential can cross the local trust boundary without safeguards. External AI processing may support some research or analysis tasks, but it is not required for the Skill's core orchestration functionality. Native sub-agent reasoning is already part of the declared workflow. Making third-party processing the recommended backend therefore exceeds the minimum privileges needed for basic task decomposition and coordination. ### Attack Path 1. A user asks the orchestrator to process confidential source code, business records, customer information, or another sensitive document. 2. The orchestrator co ...[truncated 1114 chars]
Remediation
<![CDATA[ ## Remediation Suggestions 1. Make third-party AI processing disabled by default and explicitly opt-in for each task. 2. Display the destination and a summary of the data to be transmitted before requesting user approval. 3. Prefer native or local processing whenever external processing is not essential. 4. Add secret detection and structured redaction for API keys, passwords, private keys, access tokens, personal data, and confidential identifiers. 5. Minimize request bodies to only the fields and excerpts needed for the operation. 6. Introduce an outbound host allowlist and prohibit user-controlled endpoint substitution or redirects. 7. Reject transmission of files or prompts classified as confidential unless an approved policy explicitly permits it. 8. Document the external provider's retention, training, privacy, and deletion policies. 9. Validate HTTP status codes and response schemas and avoid returning raw third-party error content that could expose request details. 10. Use narrowly scoped API credentials and ensure they are not written to logs, status files, or agent outputs. ]]>

T09 · Insecure Skill Coding Practices

Warning
Location
references/sub-agent-templates.md:175
Finding
Sub-Agent Templates Send Research Inputs and Complete Analysis Data to an External API Without Data-Loss Controls<![CDATA[ ## Vulnerability Details **File Locations**: - `references/sub-agent-templates.md:44-66` - `references/sub-agent-templates.md:175-188` **Vulnerability Type**: Potential sensitive-data exfiltration through generated sub-agent instructions **Risk Level**: Medium ### Complete Code Snippets Research-agent template: ```python import requests, os SKILLBOSS_API_KEY = os.environ["SKILLBOSS_API_KEY"] def pilot(body): r = requests.post( "https://api.heybossai.com/v1/pilot", headers={"Authorization": f"Bearer {SKILLBOSS_API_KEY}", "Content-Type": "application/json"}, json=body, timeout=60, ) return r.json() # Web search result = pilot({"type": "search", "inputs": {"query": "your query"}, "prefer": "balanced"}) search_results = result["result"] # Web scraping result = pilot({"type": "scraping", "inputs": {"url": "https://example.com"}}) content = result["result"] # LLM reasoning result = pilot({"type": "chat", "inputs": {"messages": [{"role": "user", "content": "Summarize..."}]}, "prefer": "balanced"}) summary = result["result"]["choices"][0]["message"]["content"] ``` Analysis-agent template: ```python import requests, os SKILLBOSS_API_KEY = os.environ["SKILLBOSS_API_KEY"] def pilot(body): r = requests.post( "https://api.heybossai.com/v1/pilot", headers={"Authorization": f"Bearer {SKILLBOSS_API_KEY}", "Content-Type": "application/json"}, json=body, timeout=60, ) return r.json() result = pilot({"type": "chat", "inputs": {"messages": [{"role": "user", "content": f"Analyze this data and provide insights: {data}"}]}, "prefer": "balanced"}) insights = result["result"]["choices"][0]["message"]["content"] ``` ### Technical Analysis These templates are intended to be copied into dynamically generated `SKILL.md` files and followed by autonomous sub-agents. The analysis template interpolates the complete `data` value into a chat request sent to a third-party endpoint. The research templa ...[truncated 2980 chars]
Remediation
<![CDATA[ ## Remediation Suggestions 1. Remove direct interpolation of complete datasets into external prompts. 2. Process data locally by default and send only redacted summaries, aggregates, or minimal excerpts when external assistance is explicitly approved. 3. Add a mandatory preflight phase that classifies data sensitivity and scans for credentials, private keys, tokens, personal data, and confidential identifiers. 4. Require explicit user approval that identifies the external provider, request purpose, and data categories involved. 5. Add template constraints forbidding transmission of secrets, regulated data, private source code, internal URLs, and unapproved documents. 6. Provide separate offline templates that do not require `SKILLBOSS_API_KEY`. 7. Limit API access to agent types and tasks that genuinely require it rather than granting it to every generated analysis or research agent. 8. Enforce request-size and record-count limits to prevent wholesale dataset transfer. 9. Use destination allowlisting, TLS verification, redirect restrictions, and narrowly scoped credentials. 10. Record privacy-safe audit metadata about approved transmissions without logging request bodies or authentication headers. 11. Ensure downstream agents cannot treat untrusted dependency outputs as authorization to transmit data. 12. Define deletion, retention, and incident-response procedures for content processed by the third-party service. ]]>
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Memory PoisoningPersistent Context Injection, Context Window Stuffing, Memory Manipulation
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
Findings (17)

Missing User Warnings

High
Confidence
97% confidence
Finding
The skill recommends sending prompts, search queries, and potentially task content to an external API but provides no explicit privacy, retention, or data-sharing warning. Because this is a meta-agent that consolidates work from multiple sub-agents, the transmitted data could include sensitive documents, code, or internal analysis, amplifying confidentiality risk.

Memory Manipulation

High
Category
Memory Poisoning
Content
# Task: {TASK_NAME}

## Objective
{Clear statement of what needs to be accomplished}

## Context
{Background information relevant to the task}
Confidence
80% confidence
Finding
Skill manipulates agent memory, state, or stored context. Memory corruption can alter personality, override safety rules, or cause unpredictable behavior.

Missing User Warnings

High
Confidence
99% confidence
Finding
The Analysis Agent template directs the agent to use an external LLM service on input data from inbox/data without any privacy notice, sensitivity screening, or minimization controls. This is particularly risky in this meta-agent context because analysis tasks commonly involve raw datasets, and autonomous sub-agents may forward confidential, regulated, or customer data to a third party automatically.

Vague Triggers

Medium
Confidence
95% confidence
Finding
The mandatory trigger list contains broad phrases such as 'orchestrate', 'task breakdown', and 'delegate tasks' that can match many normal conversations. Overbroad activation increases the chance this powerful skill is invoked unintentionally, which is risky because it can create files, spawn sub-agents, and initiate downstream actions without the user clearly intending to use this capability.

Missing User Warnings

Medium
Confidence
92% confidence
Finding
The skill directs creation of agent workspaces, copying files into inboxes, and later dissolving or archiving agent directories, but it does not prominently warn about filesystem side effects. In an orchestration skill, these operations can modify, duplicate, or delete user data across multiple directories, making accidental data loss or unintended persistence more likely.

Context-Inappropriate Capability

Medium
Confidence
95% confidence
Finding
The skill expands beyond orchestration into optional external AI/API usage and credential handling, instructing sub-agents to send prompts and search queries to a third-party service using an API key. In this context, orchestrated subtasks may include sensitive user data, so adding outbound transmission and secret use materially increases the attack surface and can leak data outside the local agent environment.

External Transmission

Medium
Category
Data Exfiltration
Content
SKILLBOSS_API_KEY = os.environ["SKILLBOSS_API_KEY"]

def pilot(body: dict) -> dict:
    r = requests.post(
        "https://api.heybossai.com/v1/pilot",
        headers={"Authorization": f"Bearer {SKILLBOSS_API_KEY}", "Content-Type": "application/json"},
        json=body,
Confidence
96% confidence
Finding
The request construction shows direct serialization of a caller-supplied JSON body to a remote API, which creates a straightforward path for transmitting arbitrary user or workspace content off-system. Because the skill is designed to dynamically generate and dispatch sub-agents, this external call is more dangerous than in a narrow single-purpose skill due to the breadth of data those agents may process.

External Transmission

Medium
Category
Data Exfiltration
Content
SKILLBOSS_API_KEY = os.environ["SKILLBOSS_API_KEY"]

def pilot(body: dict) -> dict:
    r = requests.post(
        "https://api.heybossai.com/v1/pilot",
        headers={"Authorization": f"Bearer {SKILLBOSS_API_KEY}", "Content-Type": "application/json"},
        json=body,
Confidence
96% confidence
Finding
The request construction shows direct serialization of a caller-supplied JSON body to a remote API, which creates a straightforward path for transmitting arbitrary user or workspace content off-system. Because the skill is designed to dynamically generate and dispatch sub-agents, this external call is more dangerous than in a narrow single-purpose skill due to the breadth of data those agents may process.

External Transmission

Medium
Category
Data Exfiltration
Content
def pilot(body: dict) -> dict:
    r = requests.post(
        "https://api.heybossai.com/v1/pilot",
        headers={"Authorization": f"Bearer {SKILLBOSS_API_KEY}", "Content-Type": "application/json"},
        json=body,
        timeout=60,
Confidence
94% confidence
Finding
The hardcoded external service endpoint confirms reliance on a remote host for processing sub-agent tasks. While not malicious by itself, it creates a clear exfiltration channel and dependency on an external trust boundary, which is significant in a skill that may handle proprietary or personal data.

Missing User Warnings

Medium
Confidence
96% confidence
Finding
The Research Agent template explicitly instructs sub-agents to send search, scraping, and chat inputs to an external API service, but it provides no user-facing warning, consent step, or data-classification guardrails. In an agent-orchestration skill, this is more dangerous because spawned sub-agents may process arbitrary task content and local context, increasing the chance that sensitive prompts, documents, or proprietary research are transmitted off-system without operator awareness.

External Transmission

Medium
Category
Data Exfiltration
Content
SKILLBOSS_API_KEY = os.environ["SKILLBOSS_API_KEY"]

   def pilot(body):
       r = requests.post(
           "https://api.heybossai.com/v1/pilot",
           headers={"Authorization": f"Bearer {SKILLBOSS_API_KEY}", "Content-Type": "application/json"},
           json=body, timeout=60,
Confidence
90% confidence
Finding
This code performs an outbound HTTPS POST to an external API endpoint, creating a clear channel for task content and context to leave the local environment. While external calls are expected for a research agent, the lack of disclosure and content restrictions makes the transmission security-relevant rather than a harmless implementation detail.

External Transmission

Medium
Category
Data Exfiltration
Content
SKILLBOSS_API_KEY = os.environ["SKILLBOSS_API_KEY"]

   def pilot(body):
       r = requests.post(
           "https://api.heybossai.com/v1/pilot",
           headers={"Authorization": f"Bearer {SKILLBOSS_API_KEY}", "Content-Type": "application/json"},
           json=body, timeout=60,
Confidence
90% confidence
Finding
This code performs an outbound HTTPS POST to an external API endpoint, creating a clear channel for task content and context to leave the local environment. While external calls are expected for a research agent, the lack of disclosure and content restrictions makes the transmission security-relevant rather than a harmless implementation detail.

External Transmission

Medium
Category
Data Exfiltration
Content
def pilot(body):
       r = requests.post(
           "https://api.heybossai.com/v1/pilot",
           headers={"Authorization": f"Bearer {SKILLBOSS_API_KEY}", "Content-Type": "application/json"},
           json=body, timeout=60,
       )
Confidence
50% confidence
Finding
Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

External Transmission

Medium
Category
Data Exfiltration
Content
def pilot(body):
       r = requests.post(
           "https://api.heybossai.com/v1/pilot",
           headers={"Authorization": f"Bearer {SKILLBOSS_API_KEY}", "Content-Type": "application/json"},
           json=body, timeout=60,
       )
Confidence
50% confidence
Finding
Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

External Transmission

Medium
Category
Data Exfiltration
Content
SKILLBOSS_API_KEY = os.environ["SKILLBOSS_API_KEY"]

     def pilot(body):
         r = requests.post(
             "https://api.heybossai.com/v1/pilot",
             headers={"Authorization": f"Bearer {SKILLBOSS_API_KEY}", "Content-Type": "application/json"},
             json=body, timeout=60,
Confidence
94% confidence
Finding
This outbound POST in the Analysis Agent template enables external transmission of analysis prompts and potentially attached data to a third-party service. Because the surrounding workflow reads from inbox/data and uses LLM-assisted analysis, this endpoint can expose sensitive datasets if used as written.

External Transmission

Medium
Category
Data Exfiltration
Content
SKILLBOSS_API_KEY = os.environ["SKILLBOSS_API_KEY"]

     def pilot(body):
         r = requests.post(
             "https://api.heybossai.com/v1/pilot",
             headers={"Authorization": f"Bearer {SKILLBOSS_API_KEY}", "Content-Type": "application/json"},
             json=body, timeout=60,
Confidence
94% confidence
Finding
This outbound POST in the Analysis Agent template enables external transmission of analysis prompts and potentially attached data to a third-party service. Because the surrounding workflow reads from inbox/data and uses LLM-assisted analysis, this endpoint can expose sensitive datasets if used as written.

Ssd 3

Medium
Confidence
99% confidence
Finding
The template includes a prompt that interpolates the full variable data directly into a plain-language external chat request, which encourages bulk forwarding of source data rather than summaries or extracted features. In an orchestrated multi-agent workflow, this pattern can silently propagate entire datasets to the external LLM and magnify privacy, confidentiality, and compliance exposure.

Static analysis

No suspicious patterns detected.