ClawHub
Back to skill

Security audit

Human Test

Security checks for vulnerabilities and agentic risk

Overview

The skill is not clearly malicious, but it asks users to install and run an unpinned package, expose AI API keys, and optionally grant repository access to an external GitHub account.

Review carefully before installing. Use the hosted flow only for products and workflows you are comfortable showing to third-party testers, and avoid sending private staging systems or sensitive user data without a data-handling agreement. For self-hosting, run it in an isolated environment with only one low-privilege API key exposed, avoid global unpinned installs, and review the package source/version first. For repo features, prefer a disposable mirror or read-only access, revoke access after use, and manually review every diff or PR before merging.

A.I.G

Vulnerability Patterns
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
  • Insecure DependenciesIntroduces malicious components through unsafe dependency sources
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Findings (3)

T08 · Insecure Dependencies

Error
Location
SKILL.md:42
Finding

Unpinned Global Installation and Immediate Execution of a Third-Party Package

Content
View full analysis

Vulnerability Details

File Location: SKILL.md, lines 42-45
Vulnerability Type: Insecure third-party dependency installation
Risk Level: High

bash
If the server is not running, install and start it:

```bash
npm i -g humantest-app
cd /tmp && humantest init --non-interactive && cd humantest && humantest start
text

### Technical Analysis

The instructions install `humantest-app` globally without pinning an audited version or verifying package integrity, and then immediately execute commands supplied by that package. An unpinned npm installation resolves the package version at execution time, meaning the effective implementation can change after the Skill has been reviewed.

npm installation may also execute package lifecycle scripts. Global installation increases the potential scope of changes and exposes the invoking account to code supplied by the current package release. The audit did not establish that the named package is malicious; the vulnerability is the unsafe dependency acquisition and execution pattern.

### Attack Path

1. An attacker compromises the npm package, its publisher account, or a future package release.
2. The attacker introduces malicious code into package lifecycle scripts or the `humantest` executable.
3. An agent follows the documented setup and runs `npm i -g humantest-app`.
4. npm downloads the current unpinned release and may execute its lifecycle scripts.
5. The agent subsequently runs `humantest init` and `humantest start`, executing the downloaded implementation.
6. The malicious implementation operates with the filesystem, network, environment-variable, and process privileges of the invoking user.

### Impact Assessment

Successful exploitation could provide arbitrary code execution with the invoking user's privileges. Depending on that account's permissions, an attacker could read accessible files and environment variables, modify user-own
...[truncated 175 chars]
Remediation
View remediation

Remediation Suggestions

  • Pin humantest-app to a specifically reviewed version rather than resolving the latest release.
  • Verify the package with an integrity hash, signed release, or trusted lockfile before execution.
  • Avoid global installation. Install the package in an isolated project or invoke it from a disposable, least-privileged container.
  • Disable npm lifecycle scripts during installation where compatible, then review any scripts that must be enabled.
  • Run the service under a dedicated account with a sanitized environment, restricted filesystem access, and outbound-network controls.
  • Document the expected package publisher, version, checksum, and update-review process.

T05 · Unauthorized Access and Privilege Escalation

Error
Location
SKILL.md:48
Finding

Automatic Discovery of Credential-Bearing Environment Variables

Content
View full analysis

Vulnerability Details

File Location: SKILL.md, line 48
Vulnerability Type: Excessive access to sensitive credentials
Risk Level: High

text
This auto-detects AI API keys from your environment (`ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, `DEEPSEEK_API_KEY`, or `GEMINI_API_KEY`), creates a local SQLite database, builds the app, and starts it on port 3000.

Technical Analysis

The installed application is explicitly expected to inspect the process environment for several credential-bearing variables. This grants the third-party application access to every listed API key inherited by the process, even if only one provider is needed.

The Skill does not specify isolation controls, selection of a single credential, redaction requirements, storage behavior, or protections against logging secrets. Because this behavior is combined with installation and execution of a third-party npm package, a compromised or defective implementation could read and disclose valuable credentials. The audited document does not prove that credentials are actually exfiltrated; the confirmed issue is unnecessarily broad credential exposure.

Attack Path

  1. A user or agent runs the setup from a shell containing one or more AI-provider API keys.
  2. The humantest initialization or startup process inherits the complete environment.
  3. The application automatically searches for the documented credential names.
  4. A compromised dependency, unsafe diagnostic routine, or application flaw captures one or more keys.
  5. The exposed credentials are logged, persisted, or transmitted to an attacker-controlled destination.
  6. An attacker uses the keys against the corresponding provider APIs until the credentials expire or are revoked.

Impact Assessment

Exposure could allow unauthorized API requests under the affected provider accounts, resulting in financial charges, quota depletion, access to provider-side resources permitted by the ke ...[truncated 164 chars]

Remediation
View remediation

Remediation Suggestions

  • Do not scan the inherited environment for multiple provider credentials.
  • Require the operator to select one provider and supply only the single credential required for that execution.
  • Launch the application with a sanitized environment that omits unrelated secrets.
  • Provide credentials through a narrowly scoped secret mount or secret manager rather than a general process environment where possible.
  • Use dedicated, low-privilege, short-lived API credentials with spending and quota limits.
  • Ensure secrets are never written to SQLite, build output, logs, crash reports, telemetry, or command history.
  • Document credential use, retention, transmission, and deletion behavior.
  • Add automated secret-redaction tests and immediately revoke any credential suspected of exposure.

T05 · Unauthorized Access and Privilege Escalation

Error
Location
SKILL.md:224
Finding

External GitHub Account Receives Repository Read or Write Access

Content
View full analysis

Vulnerability Details

File Location: SKILL.md, lines 224-230
Vulnerability Type: Excessive third-party repository permissions
Risk Level: High

markdown
If you pass a `repoUrl`, the platform automatically triggers code fix generation as soon as the report is ready. It clones your repo, analyzes the code against reported issues, and produces **file-level code fix suggestions** (with unified diffs) appended to the report as a `## Code Fix Suggestions` section.

### Two modes (auto-detected)

**Mode 1 — Read-only:** Grant GitHub user `avivahe326` read access to your repo. After the report, the platform clones the repo, analyzes the code against reported issues, and appends code-level diffs to the report.

**Mode 2 — Developer access:** Grant `avivahe326` write access. Same as Mode 1, plus: creates a branch `human-test/fixes-<taskId>`, applies the diffs, pushes, and opens a PR. The PR URL is returned in the webhook payload as `codeFixPrUrl` and in the status API.

Technical Analysis

The workflow asks users to grant a named external GitHub user access to a repository. Read access permits the external service or account to clone and analyze private source code. Developer access additionally permits automated branch creation, modification application, pushes, and pull-request creation.

This creates a substantial trust boundary and grants permissions beyond those required for the Skill's primary usability-testing function. A persistent user collaboration also lacks the fine-grained, short-lived, repository-scoped authorization model normally provided by a dedicated GitHub App. The audit found no evidence that the named account is malicious; the risk arises from broad third-party access and automated write capability.

Attack Path

  1. A repository owner follows the documentation and grants the external GitHub user read or developer access.
  2. The hosted platform clones the repository and processes its ...[truncated 1107 chars]
Remediation
View remediation

Remediation Suggestions

  • Replace personal-user collaboration with a dedicated GitHub App using fine-grained, repository-scoped permissions.
  • Default to read-only access and request write permission only for a separately confirmed code-change operation.
  • Use short-lived installation tokens and revoke repository access automatically when processing completes.
  • Require explicit owner approval before cloning private repositories and before every push or pull-request operation.
  • Generate patches for local review instead of applying and pushing changes automatically.
  • Enforce protected branches, required reviews, signed commits, status checks, and restrictions on workflow-file modifications.
  • Prevent untrusted pull requests from receiving CI secrets or privileged deployment credentials.
  • Record auditable access logs and clearly document source-code retention, subprocessors, encryption, and deletion policies.
  • Encourage repository owners to grant access only to a dedicated test repository or narrowly scoped mirror.

SkillSpector

Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
Findings (5)

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
90% confidence
Finding

The skill explicitly routes product URLs, tester recordings, AI-generated analyses, and structured reports through an external hosted platform and human testers, but it does not clearly warn about privacy, confidentiality, or data handling implications. This is dangerous because users may submit proprietary products, internal staging URLs, sensitive workflows, or personal data without informed consent about who will access or retain that information.

Content

No source excerpt is available for this finding.

External Transmission

Medium
Category
Data Exfiltration
Confidence
60% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · SKILL.md (reported line 38)May include surrounding context.

human_test() can run locally. Before creating a task, check if the server is reachable:

bash
curl -s BASE_URL/api/config

If the server is not running, install and start it:

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The self-hosted installation path says the tool auto-detects AI API keys from environment variables, but it does not warn that starting the service grants it access to valuable credentials already present in the agent or host environment. This creates a real secret-exposure risk because users may run an unreviewed package that can read and potentially misuse multiple provider API keys.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
96% confidence
Finding

The skill describes automatic repo cloning, code analysis, diff generation, branch creation, pushing changes, and opening PRs, including optional write access for an external GitHub user, without strong cautions about code integrity and supply-chain risk. This is dangerous because it normalizes granting repository access and permitting automated code modification by an external service, which could introduce malicious or unsafe changes into source control.

Content

No source excerpt is available for this finding.

External Transmission

Medium
Category
Data Exfiltration
Confidence
60% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · SKILL.md (reported line 235)May include surrounding context.

Example with repoUrl

bash
curl -X POST BASE_URL/api/skill/human-test \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://your-product.com",

Static analysis

No suspicious patterns detected.