Undeclared Tool Scope
Medium
- Category
- MCP Least Privilege
- Confidence
- 70% confidence
- Finding
Without declared permissions the skill's intent is opaque and cannot be validated.
- Content
Security audit
Agent Tests
Security checks for vulnerabilities and agentic risk
Overview
This is a local test-tracking utility that writes test data to JSON files, with some documentation accuracy issues but no evidence of hidden or malicious behavior.
Install only if you want a local JSON-backed testing scaffold. Do not store secrets, private prompts, or sensitive model outputs in tests unless you are comfortable with them being retained on disk, and remember that current test runs simulate agent output rather than executing a real agent.
Vulnerability Patterns
- Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
- Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
- Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
- Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
- Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
Vulnerability Patterns
- Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
- MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
- MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
- Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
- Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
Findings (7)
Context-Inappropriate Capability
Medium
- Category
- Not specified by scanner
- Confidence
- 94% confidence
- Finding
The skill allows its storage root to be redirected via TEST_DIR and --dir with no validation that the target stays within the workspace or an application-owned data directory. That enables reads and writes to arbitrary filesystem locations accessible to the process, which is broader than necessary for a local test-tracking tool and could overwrite unrelated files or persist sensitive data in unintended places.
- Content
Missing User Warnings
Medium
- Category
- Not specified by scanner
- Confidence
- 89% confidence
- Finding
The skill persists prompts, expected outputs, and assertions to disk without clearly warning users that potentially sensitive test content will be retained. In an agent-testing context, prompts and expected outputs can easily contain proprietary instructions, secrets, or personal data, so silent persistence increases the risk of unintended long-term exposure.
- Content
Description-Behavior Mismatch
Medium
- Category
- Not specified by scanner
- Confidence
- 95% confidence
- Finding
The manifest says the skill can define, run, and track tests for agent behavior and perform performance benchmarking. In practice, runTest never invokes an agent or model; it uses a hardcoded placeholder output, so both pass/fail results and benchmark timings measure framework overhead rather than agent behavior. This is a semantic mismatch between the described testing capability and the implemented behavior.
- Content
Missing User Warnings
Medium
- Category
- Not specified by scanner
- Confidence
- 92% confidence
- Finding
The tool stores test results along with snippets of actual and expected outputs, again without explicit notice. Output content from agent tests may include confidential model responses, internal prompts, or user data, so retaining even truncated samples can leak sensitive information to anyone with filesystem access or through backups and sync tools.
- Content
Intent-Code Divergence
Low
- Category
- Not specified by scanner
- Confidence
- 97% confidence
- Finding
The header documentation advertises a direct single-test invocation form with name, prompt, and expected arguments. The CLI dispatcher only supports '--test' together with '--list', '--run', '--add', or '--remove', and otherwise prints usage text, so the documented invocation does not actually run a test. This is an active contradiction between documented interface and code behavior.
- Content
Missing User Warnings
Low
- Category
- Not specified by scanner
- Confidence
- 78% confidence
- Finding
The benchmark function appends benchmark records to benchmarks.json, creating persistent execution metadata. While this is less sensitive than prompts or outputs, it is still a file write with no accompanying warning or disclosure.
- Content