ClawHub
Back to skill

Security audit

agent-evaluation

Security checks for vulnerabilities and agentic risk

Overview

This skill is review-worthy because it routes all evaluation chat messages to a fixed third-party API without clear privacy controls or user choice.

Install only if you are comfortable sending evaluation prompts, transcripts, model outputs, and test cases to the SkillBoss/HeyBossAI API. Redact secrets and personal or regulated data first, and avoid this skill for confidential evaluations unless that provider has been approved for your data.

A.I.G

Vulnerability Patterns
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
Findings (1)

other

Warning
Location
SKILL.md:35
Finding

Mandatory Third-Party Transmission of Potentially Sensitive Evaluation Data

Content
View full analysis
str: r = requests.post( "https://api.heybossai.com/v1/pilot", headers={"Authorization": f"Bearer {SKILLBOSS_API_KEY}", "Content-Type": "application/json"}, json={ "type": "chat", "inputs": {"messages": messages}, "prefer": "balanced" }, timeout=60, ) return r.json()["result"]["choices"][0]["message"]["content"] ``` ### Technical Analysis The Skill directs all LLM calls to the external `api.heybossai.com` service and transmits the complete `messages` collection in the HTTP request body. Agent-evaluation inputs can contain proprietary system prompts, production conversations, personal information, test fixtures, confidential model outputs, adversarial samples, or embedded credentials. The instructions do not require data classification, secret detection, redaction, user consent, or confirmation before transmission. They also do not explain the external service's retention, processing, or privacy properties. Requiring this provider for all evaluation calls exceeds the minimum network access necessary for the declared agent-evaluation functionality because evaluation could use a local model or a provider explicitly selected by the user. The `SKILLBOSS_API_KEY` is intentionally supplied to its associated service as a Bearer token. This authentication behavior is not independently evidence of credential theft. The principal risk is disclosure of the evaluation messages to a third party without adequate safeguards. ### Attack Path 1. A user installs or foll ...[truncated 1338 chars]
Remediation
View remediation

SkillSpector

Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
Findings (4)

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The skill includes code that forwards arbitrary messages to an external API but does not clearly warn that prompts, test cases, model outputs, or other potentially sensitive evaluation data will leave the local environment. In an agent-evaluation context, this is especially risky because evaluation datasets often contain proprietary prompts, internal transcripts, or production-like data, creating an avoidable data disclosure risk.

Content

No source excerpt is available for this finding.

External Transmission

Medium
Category
Data Exfiltration
Confidence
87% confidence
Finding

The request code embeds authenticated outbound transmission to api.heybossai.com, so any caller using this snippet may unintentionally export sensitive evaluation inputs to a remote service. In this skill’s context, benchmark prompts and production-monitoring samples can be especially sensitive, making the omission of safeguards more dangerous than a generic demo snippet.

Content

Scanner excerpt · SKILL.md (reported line 44)May include surrounding context.

md
SKILLBOSS_API_KEY = os.environ["SKILLBOSS_API_KEY"]

def call_llm(messages: list) -> str:
    r = requests.post(
        "https://api.heybossai.com/v1/pilot",
        headers={"Authorization": f"Bearer {SKILLBOSS_API_KEY}", "Content-Type": "application/json"},
        json={

External Transmission

Medium
Category
Data Exfiltration
Confidence
87% confidence
Finding

The request code embeds authenticated outbound transmission to api.heybossai.com, so any caller using this snippet may unintentionally export sensitive evaluation inputs to a remote service. In this skill’s context, benchmark prompts and production-monitoring samples can be especially sensitive, making the omission of safeguards more dangerous than a generic demo snippet.

Content

Scanner excerpt · SKILL.md (reported line 44)May include surrounding context.

md
SKILLBOSS_API_KEY = os.environ["SKILLBOSS_API_KEY"]

def call_llm(messages: list) -> str:
    r = requests.post(
        "https://api.heybossai.com/v1/pilot",
        headers={"Authorization": f"Bearer {SKILLBOSS_API_KEY}", "Content-Type": "application/json"},
        json={

External Transmission

Medium
Category
Data Exfiltration
Confidence
84% confidence
Finding

The hardcoded external endpoint identifies a specific third-party service that will receive request contents, confirming that data leaves the trust boundary. By itself this is not remote code execution or credential theft, but it is still a real privacy and governance concern when the skill handles agent-evaluation artifacts that may include confidential business data.

Content

Scanner excerpt · SKILL.md (reported line 45)May include surrounding context.

md
def call_llm(messages: list) -> str:
    r = requests.post(
        "https://api.heybossai.com/v1/pilot",
        headers={"Authorization": f"Bearer {SKILLBOSS_API_KEY}", "Content-Type": "application/json"},
        json={
            "type": "chat",