ClawHub
Back to skill

Security audit

Skill Evaluation

Security checks for vulnerabilities and agentic risk

Overview

The skill mostly matches its cloud evaluation purpose, but it silently sends local skills, datasets, credentials, and resource files through insecure and under-scoped workflows.

Install only if you are comfortable authenticating to the configured evaluation service and uploading local skill code, datasets, and referenced attachments. Before use, remove secrets and symlinks from selected skill folders, avoid sensitive Excel/filelist content, use a trusted network, and understand that tokens and custom model API keys are stored locally under .eval.

A.I.G

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (3)

T09 · Insecure Skill Coding Practices

Error
Location
scripts/clients/http_client.py:59
Finding

TLS certificate verification is disabled for authenticated API traffic

Content
View full analysis

Vulnerability Details

File Location: scripts/clients/http_client.py:59-76, scripts/clients/http_client.py:150-155, scripts/clients/api_client.py:72-76, scripts/clients/api_client.py:92-97, scripts/eval_skill.py:382-393, scripts/eval_skill.py:967-978, scripts/eval_skill.py:1017-1030
Vulnerability Type: Improper certificate validation
Risk Level: High

Vulnerable Code

python
class BaseHttpClient:
    def __init__(
        self,
        base_url: str = "",
        timeout: int = DEFAULT_TIMEOUT,
        max_retries: int = MAX_RETRIES,
        retry_backoff_factor: float = RETRY_BACKOFF_FACTOR,
        verify_ssl: bool = False
    ):
        self.base_url = base_url.rstrip('/') if base_url else ""
        self.timeout = timeout
        self.max_retries = max_retries
        self.retry_backoff_factor = retry_backoff_factor
        self.verify_ssl = verify_ssl
        self.session = self._create_session()
python
kwargs.setdefault("timeout", self.timeout)
kwargs.setdefault("verify", self.verify_ssl)

headers = kwargs.pop("headers", {})
headers.setdefault("Content-Type", "application/json")
headers = self._inject_auth(headers)
kwargs["headers"] = headers

resp = self.session.request(method, url, **kwargs)

The authenticated API client explicitly preserves the insecure setting:

python
super().__init__(
    base_url=base_url,
    timeout=timeout,
    verify_ssl=False
)

Bearer credentials are then attached to requests:

python
def _inject_auth(self, headers: Dict[str, str]) -> Dict[str, str]:
    token = self.token_manager.get_token()
    headers["Authorization"] = f"Bearer {token}"
    if self._username:
        headers["x-consumer-username"] = self._username
    return headers

Multipart uploads also explicitly disable validation:

python
resp = requests.post(
    f"{base_url}/eval/api/v1/skill/eval-resource/file",
    headers=_bearer_headers(auth_file),
    files=files,
    data=data_form,
    timeout=
...[truncated 2366 chars]
Remediation
View remediation

Remediation Suggestions

  1. Change the HTTP client default to certificate verification enabled:
python
verify_ssl: bool = True
  1. Remove every explicit verify=False argument from multipart upload requests.
  2. Remove global suppression of InsecureRequestWarning.
  3. If the service uses a private certificate authority, configure verify with a controlled CA bundle path instead of disabling verification.
  4. Reject non-HTTPS platform base URLs before attaching Bearer credentials.
  5. Restrict the configured service origin to an explicit allowlist before sending authorization headers.
  6. Add automated tests confirming that invalid, expired, self-signed, and hostname-mismatched certificates are rejected.
  7. Rotate potentially exposed tokens after deploying the correction.

T09 · Insecure Skill Coding Practices

Warning
Location
scripts/eval_skill.py:366
Finding

Recursive Skill packaging can upload secrets and files referenced through symlinks

Content
View full analysis

Vulnerability Details

File Location: scripts/eval_skill.py:366-373
Related Workflow Location: processes/skill-selection.md:91-99, processes/skill-selection.md:126-145, processes/skill-selection.md:207-213
Vulnerability Type: Unrestricted recursive file collection and silent data upload
Risk Level: Medium

Vulnerable Code

python
zip_path = output_dir / f"{skill_name}.zip"
included_names = {skill_dir.name}
with zipfile.ZipFile(zip_path, 'w', zipfile.ZIP_DEFLATED) as zf:
    for src in [skill_dir, *extra_dirs]:
        if src.name in included_names and src is not skill_dir:
            continue
        included_names.add(src.name)
        for file_path in src.rglob('*'):
            if file_path.is_file():
                arcname = Path(src.name) / file_path.relative_to(src)
                zf.write(file_path, arcname)

The resulting archive is immediately uploaded:

python
with open(zip_path, "rb") as f:
    files = {"file": f}
    data_form = {"path": "skills/"}
    resp = requests.post(
        f"{base_url}/eval/api/v1/skill/eval-resource/file",
        headers=_bearer_headers(auth_file),
        files=files,
        data=data_form,
        timeout=60,
        verify=False,
    )

Technical Analysis

The packaging routine recursively includes every object for which Path.is_file() returns true. It has no denylist or allowlist for sensitive content and does not inspect the resolved path before archiving it.

Consequently, the archive can include:

  • .env files and local configuration secrets.
  • API keys, tokens, credentials, or private-key files stored under the Skill directory.
  • Version-control metadata and unintended development artifacts.
  • Large generated files or caches.
  • Files outside the Skill directory referenced by symbolic links.

For a symbolic link to a regular file, is_file() can follow the link and ZipFile.write() can archive the target's content. Because there is no check that `file_path ...[truncated 1888 chars]

Remediation
View remediation

Remediation Suggestions

  1. Reject symbolic links during packaging, or resolve every candidate and verify containment:
python
root = src.resolve()
resolved = file_path.resolve(strict=True)
resolved.relative_to(root)
  1. Use os.lstat() or equivalent checks to distinguish links from regular files before reading content.
  2. Apply a default denylist for common sensitive and irrelevant paths, including .env, .git, private keys, credential files, caches, virtual environments, build output, and the local .eval directory.
  3. Prefer an explicit packaging allowlist containing only files required to run the Skill.
  4. Scan the archive manifest for likely credentials and secret-bearing file names before upload.
  5. Display the final archive manifest or a concise inclusion summary and obtain informed confirmation before network transmission.
  6. Add file-count and total-size limits to prevent accidental oversized uploads.
  7. Add tests covering file symlinks, directory symlinks, nested secret files, broken links, and paths that resolve outside the selected root.

T09 · Insecure Skill Coding Practices

Warning
Location
scripts/files/file_utils.py:68
Finding

OAuth access tokens and PKCE state are stored without explicit restrictive permissions

Content
View full analysis

Vulnerability Details

File Location: scripts/files/file_utils.py:68-84
Sensitive Call Sites: scripts/eval_auth.py:86-105, scripts/eval_auth.py:233-244
Vulnerability Type: Insecure local credential storage
Risk Level: Medium

Vulnerable Code

The generic JSON writer creates directories and writes files without explicitly setting restrictive modes:

python
def save_json(path: str, data: Any, encoding: str = "utf-8") -> Dict[str, Any]:
    p = Path(path)

    try:
        p.parent.mkdir(parents=True, exist_ok=True)
        content = json.dumps(data, indent=2, ensure_ascii=False)
        p.write_text(content, encoding=encoding)
        return result("save", "saved", f"Save succeeded: {path}", data={"path": str(p)})
    except Exception as e:
        return result("save", "error", f"Save failed: {e}", success=False)

The same writer stores OAuth state and the PKCE verifier:

python
state_data = {
    "state_token": state_token,
    "client_id": client_id
}
if pkce_verifier:
    state_data["pkce_verifier"] = pkce_verifier
save_json(state_path, state_data)

It also stores the access token:

python
auth_cache = {
    "access_token": access_token,
    "expires_in": expires_in,
    "created_at": now.isoformat(),
    "expires_at": expires_at.isoformat()
}
save_json(output_path, auth_cache)

Technical Analysis

Path.write_text() creates a file using permissions derived from the process umask. The code does not enforce owner-only permissions on either the credential files or their containing directory. It also does not reject symbolic-link destinations or use an atomic, exclusive file-creation operation.

The following security-sensitive values are affected:

  • OAuth Bearer access token.
  • OAuth state token.
  • PKCE verifier.
  • Client identifier and token lifetime metadata.

On a shared system with a permissive umask, another local user may be able to read the files. If an attacker can pre-create or replace the output ...[truncated 1758 chars]

Remediation
View remediation

Remediation Suggestions

  1. Store authentication state in a dedicated credential-writing function rather than the generic JSON writer.
  2. Create the containing credential directory with mode 0700.
  3. Create credential files atomically with owner-only mode 0600, using exclusive creation where appropriate.
  4. If an existing file is updated, validate that it is a regular file owned by the current user and is not a symbolic link.
  5. Write updates to a securely created temporary file in the same directory, apply 0600, flush and synchronize it, then atomically replace the destination.
  6. Explicitly apply restrictive permissions after replacement because an existing file may retain unsafe permissions.
  7. Avoid returning or printing full access tokens in command output.
  8. Delete stale OAuth state files after failures and timeouts, not only after successful token exchange.
  9. Add tests under permissive umasks and tests involving pre-created symbolic links.

SkillSpector

Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
Findings (90)

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

声明描述聚焦于“Skill 自动化评测/横评/对比”，而代码片段没有任何评测、报告生成、模型对比、Skill 验证或横评逻辑。相反，它是底层的认证服务客户端，实现了与 OAuth2 认证接口通信及响应处理。这属于与声明主目的明显不同的能力和资源访问方向，因此应判定为描述与实际行为不匹配。

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding

声明描述的核心能力是“自动化评测 Skill 并生成评测报告”，这通常应包含评测流程编排、结果打分/汇总、横向对比或报告生成等逻辑。但提供的代码片段仅是底层网络通信组件 BaseHttpClient，没有任何与 Skill 评测、结果分析、报告输出、多 Skill 对比或多模型对比相关的实现。虽然该 HTTP 客户端可能作为评测系统的支撑模块存在，但就此代码片段本身而言，其主要目的与声明用途明显不一致，并且暴露了未声明的通用网络访问能力。因此应判定为描述与实际行为不匹配。

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

声明描述的核心能力是“评测 Skill 效果并生成评测报告”，而代码完全没有任何评测、报告生成、横评、多模型对比或框架对比逻辑。相反，代码是典型的 OAuth 登录授权辅助组件，用于本地回调接收授权码并进行 state 校验，属于身份认证/授权流程支持代码。这不是为了实现已声明功能所必需的普通底层细节，而是一个不同领域的独立功能，因此存在明显的描述与行为不一致。

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

声明描述的是“评测/横评/对比 Skill 表现”的能力，而实际代码片段仅涉及认证令牌管理，与评测逻辑、报告生成、模型对比或框架对比无直接关系。这不是单纯的底层支撑细节可以合理解释的范围，因为该代码的主功能明显是身份认证状态管理，而非评测执行。因此描述与代码行为存在明显不匹配。

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

声明描述的核心能力是“评测 skill 并输出评测报告”，属于评测执行与结果分析类功能；但代码并未实现任何 skill 评测、横评、多模型对比、框架对比或报告生成逻辑。相反，代码专注于评测服务的身份认证与令牌管理，包括自动/手动登录、浏览器回调、PKCE、state 校验、token 缓存和检查。这些能力虽然可能是评测系统的配套基础设施，但就该代码块本身而言，其主要目的与声明的 Skill 评测功能明显不一致，因此应判定为描述与行为不匹配。

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

声明描述的是“Skill 自动化评测/横评/对比”功能，但代码片段仅是一个文件处理模块的 init.py，对外导出若干通用文件与数据加载能力，没有体现任何评测流程、报告生成、Skill 验证、模型对比或框架对比逻辑。虽然文件处理可能作为评测系统的底层支撑模块，但就该代码片段本身而言，其直接功能与声明的主要用途明显不一致，因此应判定为描述与实际行为不匹配。

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
98% confidence
Finding

声明描述的核心能力应是“自动化评测 Skill 并生成评测报告/对比结果”。但该代码片段没有体现评测流程、执行 Skill、模型对比、框架对比、打分或报告生成等行为；它主要是底层文件工具和数据预处理/配置加载模块。虽然这类工具可能作为评测系统的辅助组件存在，但就该代码片段本身而言，其实际功能与声明的主要用途不一致，且包含未在描述中明确体现的通用文件处理与字段映射能力。因此应判定为描述与代码行为不匹配。

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

声明描述的是一个“Skill 自动化评测”能力，核心应涉及评测流程、比较分析、报告输出等。而提供的代码仅是通用文件读取模块，负责从本地路径读取 JSONL/CSV 文件并逐条产出记录，不包含任何评测逻辑、报告生成、模型对比、Skill 验证或触发词相关实现。因此代码的主要用途与声明严重不符，且实际具备了未声明的本地文件读取能力。应判定为描述与行为不匹配。

Content

No source excerpt is available for this finding.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding

声明描述的核心能力是“自动化评测 Skill 并生成评测报告”，但实际代码仅实现了通用日期时间工具函数（ISO 时间解析、过期判断）。这不是评测功能的直接体现，也看不出与单 Skill 验证、多 Skill 横评、多模型对比、框架对比等声明能力有关。代码片段没有访问敏感资源或额外权限问题，但其实际用途与声明的主要目的明显不一致，因此应判定为描述与行为不匹配。

Content

No source excerpt is available for this finding.

Missing User Warnings

High
Category
Not specified by scanner
Confidence
96% confidence
Finding

The workflow instructs the agent to upload a user-provided dataset to the cloud as a background step while explicitly suppressing any user-facing notice that remote transmission is occurring. This creates a meaningful privacy and data-governance risk because Excel content and referenced file paths may contain sensitive business data, and the user is denied an informed chance to consent, redact, or choose a local-only path.

Content

No source excerpt is available for this finding.

Missing User Warnings

High
Category
Not specified by scanner
Confidence
95% confidence
Finding

The workflow instructs the agent to perform deletion of dataset cases in the cloud and then remove corresponding local entries, all silently, after only collecting raw case IDs. This is dangerous because destructive actions occur without an explicit confirmation step, clear warning of scope, or user-visible execution details, increasing the risk of accidental or unauthorized data loss.

Content

No source excerpt is available for this finding.

Missing User Warnings

High
Category
Not specified by scanner
Confidence
96% confidence
Finding

The process tells the user to edit a local dataset file and then silently performs a full cloud overwrite using upload-dataset, replacing the remote dataset without a final warning or confirmation. Full replacement is especially risky because a mistaken local edit, stale file, or manipulated content can overwrite the entire remote dataset and cause broad integrity loss.

Content

No source excerpt is available for this finding.

Missing User Warnings

High
Category
Not specified by scanner
Confidence
98% confidence
Finding

The client disables TLS certificate verification by default (verify_ssl=False) for an authentication-related HTTP client. That makes the client vulnerable to man-in-the-middle interception or modification of OAuth2 credentials, tokens, and error flows, especially because this code is explicitly used for authentication service communication.

Content

No source excerpt is available for this finding.

Missing User Warnings

High
Category
Not specified by scanner
Confidence
99% confidence
Finding

Authenticated file uploads are performed with verify=False, which disables TLS certificate validation. This allows a man-in-the-middle attacker to intercept or modify uploaded skill archives and Bearer tokens, potentially stealing credentials or substituting malicious content during evaluation workflows.

Content

No source excerpt is available for this finding.

Missing User Warnings

High
Category
Not specified by scanner
Confidence
99% confidence
Finding

This upload path also disables TLS certificate verification while sending authenticated requests. An active network attacker could capture the Bearer token, tamper with responses, or redirect the upload flow, compromising both credentials and the integrity of evaluated artifacts.

Content

No source excerpt is available for this finding.

Missing User Warnings

High
Category
Not specified by scanner
Confidence
99% confidence
Finding

Resource file uploads occur over HTTPS with certificate checks disabled, exposing both uploaded files and authorization headers to interception or modification. Because this function is meant for user-supplied evaluation resources, it can leak sensitive local documents as well as credentials when used on hostile networks.

Content

No source excerpt is available for this finding.

Missing User Warnings

High
Category
Not specified by scanner
Confidence
99% confidence
Finding

During Excel conversion, referenced files may be uploaded with TLS verification disabled, again exposing file contents and Bearer credentials to man-in-the-middle attacks. Because filelist entries may point to sensitive local documents, this can result in confidentiality loss beyond the evaluation data itself.

Content

No source excerpt is available for this finding.

Undeclared Tool Scope

Medium
Category
MCP Least Privilege
Confidence
90% confidence
Finding

The skill advertises and instructs actions that imply broad capabilities including authentication, file access, networking, packaging, upload, and polling, but it does not declare any explicit tool scope or permission boundaries. This creates an over-privileged and opaque execution model where reviewers and users cannot tell what resources the skill is allowed to access, increasing the risk of unintended file access, data exfiltration, or command execution.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

The trigger phrases include broad natural-language expressions such as '测一下这个 skill' and '验证 skill 表现', which can cause the skill to activate in conversations where the user did not intend to start an evaluation workflow. Unintended activation is risky here because the skill includes silent authentication and backend actions, so accidental invocation could lead to unexpected data access, packaging, or network activity.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
97% confidence
Finding

L42 明确要求“全程输出中文”，这是对语言/locale 的强制约束，但文档中没有提供用户选择、显式 opt-in，亦未说明该技能仅面向特定中文区域或合规场景。根据语言与 locale 政策，这类无选择的强制语言限制属于自然语言策略违规。

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

The skill content is written to enforce Chinese-language user-facing interaction without presenting a language choice or opt-in path. In an evaluation workflow, this can mislead users, reduce informed consent, and cause operators to misunderstand model/runtime selections or confirmation steps, especially where precise configuration matters.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
97% confidence
Finding

The file presents all operational headings and required user-facing output in Chinese, including prescribed prompts such as the natural-language guidance and authorization instructions. There is no indication that the user can choose another language or that the locale restriction is required for a region-specific purpose.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

The skill explicitly requires silent background token validation and login, while storing credentials in a local auth.json file, without any user-facing disclosure or consent step. This creates a real security and privacy risk because users may be unaware that authentication actions are occurring or that sensitive tokens are being persisted locally, increasing the chance of unintended credential exposure or misuse.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

The skill is written entirely in Chinese and includes prescribed user-facing prompts and labels in Chinese, but does not indicate that language selection is optional or user-configurable. This creates a locale policy concern because the skill appears to require a specific language for interaction without documented opt-in or justification.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
87% confidence
Finding

The skill instructions and required user-facing messages are written entirely in Chinese, including mandated output text, with no indication that the user can choose another language. This creates a language/locale policy concern because the skill appears to impose a specific language without opt-in or justification.

Content

No source excerpt is available for this finding.

Static analysis

Detected: suspicious.insecure_tls_verification

HTTPS certificate verification is disabled.

Warn
Code
suspicious.insecure_tls_verification
Location
scripts/eval_skill.py:391