Caveman

• Outcome: Pass
• Latest audit: Sep 16, 2026 · 5:11 AM UTC
• Version: 1.0.2

Overview

This is a disclosed response-style skill with some usability and supply-chain cautions, but no evidence of hidden data access, destructive behavior, or malicious instructions.

Before installing, prefer inspecting or pinning the source/installer instead of running mutable npx or plugin commands blindly. Use this skill for casual concise output, and switch to normal mode for medical, legal, financial, security, or other nuanced work where caveats and uncertainty matter. Run the benchmark script only in an isolated environment with the intended Anthropic credentials exposed.

A.I.G

Vulnerability Patterns

• Insecure Dependencies: Introduces malicious components through unsafe dependency sources
• Skill Instruction Hijacking: Alters the agent's session goals or safety constraints when the skill loads
• Agent Memory Poisoning: Writes attacker-controlled rules into memory that affect later sessions
• Remote Payload Retrieval and Execution: Fetches external code whose behavior can change after review
• Embedded Malicious Code: Ships malicious scripts inside the skill and executes them locally

Findings (2)

T08 · Insecure Dependencies

Warning

• Location: README.md:103

Finding

Unpinned npx Installer and Mutable Repository Reference

Content

View full analysis

Remediation

View remediation

add ``` - Pin the repository content to an immutable commit hash or signed release tag rather than the moving default branch. - Publish and document cryptographic checksums or signature-verification procedures for release artifacts. - Avoid recommending elevated execution such as `sudo`. - Document what files the installer creates or modifies and what network destinations it contacts. - Where supported, provide a manual installation method that lets users inspect the Skill files before activation. - Add automated release checks that verify the published package and repository content correspond to the audited source. ]]>

T08 · Insecure Dependencies

Note

• Location: benchmarks/requirements.txt:1

Finding

Unbounded Anthropic SDK Dependency

Content

View full analysis

=0.40.0 ``` ### Technical Analysis The dependency declaration permits any Anthropic SDK release at or above version `0.40.0`. It has no exact version, upper bound, or package hash. A future installation may therefore resolve to code that was not present during this audit. This reduces build reproducibility and increases exposure to a future compromised, malicious, or incompatible package release. Python packages can execute code during installation depending on their distribution mechanism, and the SDK is imported and executed by `benchmarks/run.py`. The audited dependency name is legitimate, and no evidence shows that the current Anthropic SDK is malicious. This finding concerns unsafe dependency pinning rather than a confirmed compromise. ### Attack Path 1. A future allowed release is compromised through publisher-account theft, package-index compromise, or upstream build-system compromise. 2. A user installs the benchmark dependencies using the supplied requirements file. 3. The package manager selects the compromised release because it satisfies `>=0.40.0`. 4. Malicious code executes during package installation or when `benchmarks/run.py` imports the `anthropic` package. 5. The malicious package operates with the benchmark user's permissions and may access resources available to that process. ### Impact Assessment A compromised dependency could access user-readable project files, process environment variables, and network resources. When the benchmark runs, the process is expected to have access to Anthropic API credentials loaded from the environment or `.env.local`, so a malicious SDK release could potentially read or misuse those credentials. The dependency does not inherently grant elevated system privileges. Its scope is limited to the p ...[truncated 85 chars]

Remediation

View remediation

``` - Generate a lock file containing hashes for the approved package and all transitive dependencies. - Install with hash enforcement, such as `pip install --require-hashes`. - Use an isolated virtual environment with only the permissions required to run benchmarks. - Review dependency updates before changing the pinned version. - Add automated vulnerability, provenance, and package-integrity checks to the dependency update process. - Avoid exposing unrelated credentials or sensitive environment variables to the benchmark process. ]]>

SkillSpector

Vulnerability Patterns

• Prompt Injection: Instruction Override, Hidden Instructions, Exfiltration Commands
• Data Exfiltration: External Transmission, Env Variable Harvesting, File System Enumeration
• Privilege Escalation: Excessive Permissions, Sudo/Root Execution, Credential Access
• Supply Chain: Unpinned Dependencies, External Script Fetching, Obfuscated Code
• Trigger Abuse: Overly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger

Findings (20)

Hidden Instructions

High

• Category: Prompt Injection
• Confidence: 70% confidence

Finding

Hidden instructions were detected in comments or invisible text. These could contain malicious directives. Manual review is recommended.

Content

Scanner excerpt · README.md (reported line 73)May include surrounding context.

md

Real token counts from the Claude API ([reproduce it yourself](benchmarks/)):

<!-- BENCHMARK-TABLE-START -->
| Task | Normal (tokens) | Caveman (tokens) | Saved |
|------|---------------:|----------------:|------:|
| Explain React re-render bug | 1180 | 159 | 87% |

Credential Access

High

• Category: Privilege Escalation
• Confidence: 60% confidence

Finding

Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Content

Scanner excerpt · benchmarks/run.py (reported line 16)May include surrounding context.

python

import anthropic

# Load .env.local from repo root if it exists
_env_file = Path(__file__).parent.parent / ".env.local"
if _env_file.exists():
    for line in _env_file.read_text().splitlines():

Credential Access

High

• Category: Privilege Escalation
• Confidence: 60% confidence

Finding

Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Content

Scanner excerpt · benchmarks/run.py (reported line 17)May include surrounding context.

python

import anthropic

# Load .env.local from repo root if it exists
_env_file = Path(__file__).parent.parent / ".env.local"
if _env_file.exists():
    for line in _env_file.read_text().splitlines():
        line = line.strip()

Vague Triggers

High

• Category: Not specified by scanner
• Confidence: 97% confidence

Finding

The trigger phrases are broad enough to activate on common requests like 'be brief' or 'less tokens', which can cause the skill to override normal response behavior without a clear, explicit user opt-in. While this is not code-execution risk, it creates prompt-routing ambiguity and can unexpectedly alter how safety-critical or nuanced answers are delivered.

Content

No source excerpt is available for this finding.

Skill Enumeration

Medium

• Category: Agent Snooping
• Confidence: 80% confidence

Finding

Skill enumerates or reads other installed skills. Access to other skills' SKILL.md files or the skills directory reveals prompt instructions, capabilities, and secrets that should be invisible to peer skills.

Content

Scanner excerpt · CONTRIBUTING.md (reported line 8)May include surrounding context.

md

## How

1. Fork repo
2. Edit `skills/caveman/SKILL.md`
3. Open PR with:
   - **Before:** what caveman say now
   - **After:** what caveman say with change

Natural-Language Policy Violations

Medium

• Category: Not specified by scanner
• Confidence: 88% confidence

Finding

The skill is described as making the agent "talk like caveman," i.e., enforcing a compressed response style that removes hedging, pleasantries, and articles. In normal productivity use this may be harmless, but in security, medical, legal, or operational contexts forced brevity can omit caveats, uncertainty, or safety-critical detail unless the user explicitly opts in.

Content

No source excerpt is available for this finding.

Rp1

Medium

• Category: MCP Rug Pull
• Confidence: 70% confidence

Finding

npx commands without a version suffix (e.g. @1.0.0) create a rug-pull risk if the upstream server is compromised and publishes a malicious update.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium

• Category: Not specified by scanner
• Confidence: 96% confidence

Finding

The skill can be activated by broad natural-language phrases like "talk like caveman" and "less tokens please," which may collide with ordinary conversation and trigger behavior unintentionally. In an agent setting, accidental activation can alter future outputs without clear user intent, affecting reliability and potentially suppressing important nuance in security or safety responses.

Content

No source excerpt is available for this finding.

External Transmission

Medium

• Category: Data Exfiltration
• Confidence: 50% confidence

Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · README.md (reported line 179)May include surrounding context.

md

If caveman save you mass token, mass money — leave mass star. ⭐

[![Star History Chart](https://api.star-history.com/svg?repos=JuliusBrussee/caveman&type=Date)](https://star-history.com/#JuliusBrussee/caveman&Date)

## Also by Julius Brussee

Vague Triggers

Medium

• Category: Not specified by scanner
• Confidence: 97% confidence

Finding

The trigger list is broad enough to match common user requests such as 'be brief' or 'less tokens', which can activate the skill without clear, explicit selection. This can unexpectedly alter assistant behavior and response style, reducing predictability and potentially interfering with safety, clarity, or other higher-priority instructions in contexts where concise phrasing is not appropriate.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium

• Category: Not specified by scanner
• Confidence: 83% confidence

Finding

The skill mandates a compressed communication style globally once triggered, without checking user language preference, accessibility needs, or context sensitivity. This can degrade clarity, increase misunderstanding, and interfere with safe communication in complex or regulated topics where precision and complete phrasing matter.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium

• Category: Not specified by scanner
• Confidence: 97% confidence

Finding

The trigger list is broad enough to activate on common requests like 'be brief' or 'less tokens', which can switch the assistant into a constrained persona without clear, explicit consent. This is not code-execution dangerous, but it can degrade response quality, omit nuance, and cause behavior changes the user did not specifically request.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium

• Category: Not specified by scanner
• Confidence: 89% confidence

Finding

The skill is designed to enforce a communication style automatically once its broad trigger conditions are met, which overrides normal interaction expectations without strong opt-in. In security-sensitive or high-precision contexts, forced terse phrasing can suppress clarifying language and make important caveats easier to miss.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium

• Category: Not specified by scanner
• Confidence: 96% confidence

Finding

The skill auto-triggers on broad phrases such as 'be brief', 'less tokens', and 'token efficiency is requested', which are common user requests that do not necessarily indicate consent to a persistent persona or response-mode change. This can cause unintended activation, override user expectations, and alter assistant behavior in unrelated contexts, creating prompt-routing and policy-compliance risk rather than a direct code-execution issue.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium

• Category: Not specified by scanner
• Confidence: 86% confidence

Finding

The skill defines a mandatory response style ('Respond like smart caveman') that can be applied automatically, which risks changing output format without clear contemporaneous user consent. In safety-sensitive or professional contexts, forced terse phrasing and removal of hedging can reduce clarity, omit nuance, and interfere with normal assistant behavior even if the underlying technical content remains accurate.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low

• Category: Not specified by scanner
• Confidence: 92% confidence

Finding

The skill auto-enables a constrained speaking style based on broad efficiency-related requests rather than explicit consent to this exact mode. While not directly malicious, this can cause unintended persistent style changes that degrade usability, omit nuance, or make safety-critical communication less clear.

Content

No source excerpt is available for this finding.

Vague Triggers

Low

• Category: Not specified by scanner
• Confidence: 82% confidence

Finding

This JSON manifest contains a list of natural-language prompts but does not define how or when they should activate, nor does it provide any scope limits or negative examples. If these prompts are used as invocation examples or matching triggers, several are phrased like ordinary user requests and could overlap with normal conversation.

Content

No source excerpt is available for this finding.

Unpinned Dependencies

Low

• Category: Supply Chain
• Confidence: 97% confidence

Finding

The dependency is specified as 'anthropic>=0.40.0', which allows any future version to be installed and makes builds non-reproducible. This increases supply-chain risk because a later vulnerable or malicious release could be pulled in without review, and it also prevents verifying whether a known-safe version is consistently used.

Content

Scanner excerpt · benchmarks/requirements.txt (reported line 1)May include surrounding context.

text

anthropic>=0.40.0

Unverifiable Dependency: anthropic has 4 known advisory(ies) (CVE-2026-34450 (Claude SDK for Python has Insecure Default File Permissions in Local Filesystem ); CVE-2026-34452 (Claude SDK for Python: Memory Tool Path Validation Race Condition Allows Sandbox); CVE-2026-34450 (The Claude SDK for Python provides access to the Claude API from Python applicat) +1 more), but the manifest does not pin a version, so it is unknown whether the installed release is affected

Low

• Category: Supply Chain
• Confidence: 90% confidence

Finding

The manifest does not pin the anthropic package, and that package has known advisories, so it is impossible to determine from this file whether an affected version may be installed. In this context, the unbounded lower-bound requirement makes the dependency state unverifiable and can expose the environment to known vulnerable releases depending on resolver behavior and deployment timing.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Low

• Category: Not specified by scanner
• Confidence: 78% confidence

Finding

The module docstring says the script benchmarks 'caveman vs normal Claude output token counts', which describes measurement but omits that the code can also persist raw benchmark data to disk and modify README.md when invoked with --update-readme. That is not just missing implementation detail: the script documentation presents a pure benchmarking role while the code includes repository-modifying side effects.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.