ClawHub
Back to skill

Security audit

EvalLayer Evaluator

Security checks for vulnerabilities and agentic risk

Overview

This skill is a disclosed wrapper for sending deliverables to EvalLayer for remote evaluation; its main risk is third-party handling of submitted content, not hidden or destructive behavior.

Install only if you are comfortable sending the evaluated topic and deliverable text to EvalLayer and having content stored for aggregation. Do not submit secrets, regulated data, private customer data, or proprietary drafts unless your organization has approved that service. Use a dedicated API key and treat payout recommendations as advisory, especially because the implementation is fixed to crypto_research evaluation.

A.I.G

Vulnerability Patterns
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs

SkillSpector

Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
Findings (20)

Tp4

High
Category
MCP Tool Poisoning
Confidence
94% confidence
Finding

The code is related to the declared domain of deliverable evaluation through EvalLayer, so the overall theme matches. However, the description overstates what this code chunk actually does. The script is only a thin client wrapper around a specific demo API call: it takes two CLI arguments, escapes them, and sends them to https://api.evallayer.ai/demo/evaluate. It does not implement factual-claim extraction, quality scoring, structured verdict construction, pass/fail logic, confidence scoring, or payout recommendation logic locally. Those outcomes may come from the remote service, but they are not evidenced in the code itself. Additionally, the script is specialized to a fixed "crypto_research" task type and uses a demo endpoint with explicit rate limits, making the actual behavior materially narrower than the declared generic quality-gate capability.

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
90% confidence
Finding

The code broadly aligns with the idea of sending a deliverable to EvalLayer for evaluation, so the core integration claim is directionally correct. However, the declared description is more specific and broader in scope than what the code demonstrates. The script is hard-coded to task_type "crypto_research", which is a narrower purpose than general deliverable evaluation. Also, the stated outputs—factual claim extraction, quality scoring, structured JSON verdicts with pass/fail, confidence, and payout recommendations—are not implemented or verified by the code; it simply submits inputs to the API and returns whatever the service responds with. No undeclared file access or extra endpoints are used, and the environment variable usage is consistent with the manifest.

Content

No source excerpt is available for this finding.

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 40)May include surrounding context.

md
bash scripts/evaluate.sh "topic" "deliverable content"

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 49)May include surrounding context.

md
bash scripts/evaluate.sh "topic" "deliverable content"

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 61)May include surrounding context.

md
bash scripts/demo.sh "topic" "deliverable content"

External Script Fetching

High
Category
Supply Chain
Confidence
90% confidence
Finding

Remote code is downloaded and executed. This bypasses code review and could introduce malicious code.

Content

Scanner excerpt · scripts/evaluate.sh (reported line 16)May include surrounding context.

sh
DELIVERABLE="${2:?Usage: evaluate.sh \"topic\" \"deliverable content\"}"

if [ -z "${EVALLAYER_API_KEY:-}" ]; then
  echo '{"error": "EVALLAYER_API_KEY environment variable is not set. Register at: curl -X POST https://api.evallayer.ai/register -H \"Content-Type: application/json\" -d {\"agent_id\": \"your-id\"}"}'
  exit 1
fi

Undeclared Tool Scope

Medium
Category
MCP Least Privilege
Confidence
78% confidence
Finding

The skill advertises shell-based usage (bash, curl, python3) but does not declare an explicit tool scope such as permissions or allowed-tools. In an agent environment, this can cause the skill to run with broader shell capability than callers expect, increasing the chance of unintended command execution or misuse of local environment access.

Content

No source excerpt is available for this finding.

External Transmission

Medium
Category
Data Exfiltration
Confidence
95% confidence
Finding

The skill is explicitly designed to send user-supplied deliverable content and an API key to an external third-party service. This is dangerous when used as a workflow gate because sensitive research, internal documents, or proprietary agent outputs may be exfiltrated outside the local trust boundary and stored for downstream aggregation.

Content

Scanner excerpt · SKILL.md (reported line 9)May include surrounding context.

md
requires.env:
    - EVALLAYER_API_KEY
  requires.bins:
    - curl
    - python3
  primaryEnv: EVALLAYER_API_KEY
---

External Transmission

Medium
Category
Data Exfiltration
Confidence
92% confidence
Finding

The registration flow transmits identifying agent information to an external endpoint, creating a trust dependency on a remote service and exposing metadata about the user or agent. While expected for API enrollment, it still constitutes external transmission and should be treated as a privacy and supply-chain risk in agent environments.

Content

Scanner excerpt · SKILL.md (reported line 24)May include surrounding context.

  1. Register for a free API key:
    bash
    curl -s -X POST https://api.evallayer.ai/register \
  -H "Content-Type: application/json" \
  -d '{"agent_id": "your-agent-id"}'

External Transmission

Medium
Category
Data Exfiltration
Confidence
98% confidence
Finding

This endpoint submits arbitrary deliverable content to a remote API for evaluation, and the note explicitly says the curl-only form does not safely escape special characters. Beyond confidentiality concerns, this increases the risk of malformed JSON, accidental leakage of embedded secrets, and unsafe handling of untrusted content in shell-driven workflows.

Content

Scanner excerpt · SKILL.md (reported line 71)May include surrounding context.

For environments without python3, use curl directly:

bash
curl -s -X POST https://api.evallayer.ai/evaluate \
  -H "Authorization: Bearer $EVALLAYER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"task_type": "crypto_research", "topic": "your topic", "deliverable": "content to evaluate"}'

External Transmission

Medium
Category
Data Exfiltration
Confidence
84% confidence
Finding

Fetching detailed evaluation breakdowns from a third-party service extends the data exposure surface because claim-level analysis and notes may reveal sensitive internal reasoning or proprietary content derived from previous submissions. In a multi-agent workflow, this can leak more context than the top-level pass/fail result alone.

Content

Scanner excerpt · SKILL.md (reported line 107)May include surrounding context.

Retrieve the full claim-by-claim breakdown for any evaluation:

bash
curl -s https://api.evallayer.ai/evaluate/EVALUATION_ID

Returns each extracted claim with type, support status, confidence score, and notes.

External Transmission

Medium
Category
Data Exfiltration
Confidence
80% confidence
Finding

The intelligence endpoint requires the API key and pulls aggregated remote data into the agent workflow, creating an external dependency and exposing credentialed access to a third-party service. In practice this expands the skill beyond simple evaluation into remote intelligence consumption, which may not be expected in tightly controlled environments.

Content

Scanner excerpt · SKILL.md (reported line 125)May include surrounding context.

Access aggregated market intelligence from all evaluations:

bash
curl -s https://api.evallayer.ai/intelligence \
  -H "Authorization: Bearer $EVALLAYER_API_KEY"

External Transmission

Medium
Category
Data Exfiltration
Confidence
90% confidence
Finding

The hardcoded remote endpoint confirms that the skill is designed to transmit workflow data to api.evallayer.ai. In an agent setting, this is materially risky because users may pass proprietary drafts, internal research, or regulated data as the deliverable, and the script provides no gating, warning, or minimization beyond basic transport security.

Content

Scanner excerpt · scripts/demo.sh (reported line 19)May include surrounding context.

sh
TOPIC_ESCAPED=$(printf '%s' "$TOPIC" | python3 -c 'import json,sys; print(json.dumps(sys.stdin.read()), end="")')
DELIVERABLE_ESCAPED=$(printf '%s' "$DELIVERABLE" | python3 -c 'import json,sys; print(json.dumps(sys.stdin.read()), end="")')

curl -s -X POST "https://api.evallayer.ai/demo/evaluate" \
  -H "Content-Type: application/json" \
  -d "{
    \"task_type\": \"crypto_research\",

External Transmission

Medium
Category
Data Exfiltration
Confidence
90% confidence
Finding

The hardcoded remote endpoint confirms that the skill is designed to transmit workflow data to api.evallayer.ai. In an agent setting, this is materially risky because users may pass proprietary drafts, internal research, or regulated data as the deliverable, and the script provides no gating, warning, or minimization beyond basic transport security.

Content

Scanner excerpt · scripts/demo.sh (reported line 19)May include surrounding context.

sh
TOPIC_ESCAPED=$(printf '%s' "$TOPIC" | python3 -c 'import json,sys; print(json.dumps(sys.stdin.read()), end="")')
DELIVERABLE_ESCAPED=$(printf '%s' "$DELIVERABLE" | python3 -c 'import json,sys; print(json.dumps(sys.stdin.read()), end="")')

curl -s -X POST "https://api.evallayer.ai/demo/evaluate" \
  -H "Content-Type: application/json" \
  -d "{
    \"task_type\": \"crypto_research\",

External Transmission

Medium
Category
Data Exfiltration
Confidence
60% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · SKILL.md (reported line 117)May include surrounding context.

md
DELIVERABLE="${2:?Usage: evaluate.sh \"topic\" \"deliverable content\"}"

if [ -z "${EVALLAYER_API_KEY:-}" ]; then
  echo '{"error": "EVALLAYER_API_KEY environment variable is not set. Register at: curl -X POST https://api.evallayer.ai/register -H \"Content-Type: application/json\" -d {\"agent_id\": \"your-id\"}"}'
  exit 1
fi

External Transmission

Medium
Category
Data Exfiltration
Confidence
60% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · scripts/evaluate.sh (reported line 16)May include surrounding context.

sh
DELIVERABLE="${2:?Usage: evaluate.sh \"topic\" \"deliverable content\"}"

if [ -z "${EVALLAYER_API_KEY:-}" ]; then
  echo '{"error": "EVALLAYER_API_KEY environment variable is not set. Register at: curl -X POST https://api.evallayer.ai/register -H \"Content-Type: application/json\" -d {\"agent_id\": \"your-id\"}"}'
  exit 1
fi

External Transmission

Medium
Category
Data Exfiltration
Confidence
70% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · scripts/evaluate.sh (reported line 16)May include surrounding context.

sh
DELIVERABLE="${2:?Usage: evaluate.sh \"topic\" \"deliverable content\"}"

if [ -z "${EVALLAYER_API_KEY:-}" ]; then
  echo '{"error": "EVALLAYER_API_KEY environment variable is not set. Register at: curl -X POST https://api.evallayer.ai/register -H \"Content-Type: application/json\" -d {\"agent_id\": \"your-id\"}"}'
  exit 1
fi

External Transmission

Medium
Category
Data Exfiltration
Confidence
95% confidence
Finding

This URL is part of an actual HTTPS POST that transmits user-provided content and an API credential to an external service. In the context of an evaluation skill, this is expected behavior, but it is still a meaningful security concern because sensitive deliverables may be disclosed outside the trust boundary.

Content

Scanner excerpt · scripts/evaluate.sh (reported line 24)May include surrounding context.

sh
TOPIC_ESCAPED=$(printf '%s' "$TOPIC" | python3 -c 'import json,sys; print(json.dumps(sys.stdin.read()), end="")')
DELIVERABLE_ESCAPED=$(printf '%s' "$DELIVERABLE" | python3 -c 'import json,sys; print(json.dumps(sys.stdin.read()), end="")')

curl -s -X POST "https://api.evallayer.ai/evaluate" \
  -H "Authorization: Bearer ${EVALLAYER_API_KEY}" \
  -H "Content-Type: application/json" \
  -d "{

External Transmission

Medium
Category
Data Exfiltration
Confidence
95% confidence
Finding

This URL is part of an actual HTTPS POST that transmits user-provided content and an API credential to an external service. In the context of an evaluation skill, this is expected behavior, but it is still a meaningful security concern because sensitive deliverables may be disclosed outside the trust boundary.

Content

Scanner excerpt · scripts/evaluate.sh (reported line 24)May include surrounding context.

sh
TOPIC_ESCAPED=$(printf '%s' "$TOPIC" | python3 -c 'import json,sys; print(json.dumps(sys.stdin.read()), end="")')
DELIVERABLE_ESCAPED=$(printf '%s' "$DELIVERABLE" | python3 -c 'import json,sys; print(json.dumps(sys.stdin.read()), end="")')

curl -s -X POST "https://api.evallayer.ai/evaluate" \
  -H "Authorization: Bearer ${EVALLAYER_API_KEY}" \
  -H "Content-Type: application/json" \
  -d "{

Description-Behavior Mismatch

Low
Category
Not specified by scanner
Confidence
91% confidence
Finding

The manifest describes a general-purpose deliverable evaluation skill, but the implementation always submits requests with task_type set to "crypto_research". This means the code does not actually perform topic-agnostic evaluation as described, and is instead specialized to a specific evaluation domain.

Content

No source excerpt is available for this finding.