Back to skill

Security audit

API Test Case Generator

Security checks for vulnerabilities and agentic risk

Overview

This is a coherent API test generator, but it has real file-scope bugs that can let a malicious API spec read local YAML-like files or write generated YAML outside the chosen output folder.

Review before installing, especially if you may process API specs from other teams or external sources. Use it only on trusted specs, write outputs to a disposable directory, and avoid feeding generated LLM prompts to an agent with file/shell/network tools until the $ref containment, filename sanitization, and prompt-injection warnings are fixed.

Vulnerability Patterns
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Findings (3)

T05 · Unauthorized Access and Privilege Escalation

Error
Location
api_case_gen.py:302
Finding

Unrestricted Local File Read Through Specification-Controlled $ref Paths

Content
View full analysis
dict: key = str(yaml_path.resolve()) if key not in self._cache: if yaml_path.exists(): try: with open(yaml_path, encoding="utf-8") as f: self._cache[key] = yaml.safe_load(f) or {} except yaml.YAMLError as e: log.error("YAML parse error in %s: %s", yaml_path, e) self._cache[key] = {} else: log.debug("File not found: %s", yaml_path) self._cache[key] = {} return self._cache[key] def resolve_ref(self, ref: str, context_file: Path) -> tuple[dict, str, Path]: """Resolve a $ref string relative to context_file.""" if "#" in ref: file_part, fragment = ref.split("#", 1) else: file_part, fragment = ref, "" if file_part: target_path = (context_file.parent / file_part).resolve() else: target_path = context_file.resolve() if not target_path.exists() and self._root and file_part: real_parts = [p for p in Path(file_part).parts if p != ".."] if len(real_parts) >= 2: module = real_parts[0] rest = Path(*real_parts[1:]) for vdir in self._version_dirs: candidate = self._root / module / vdir / rest if candidate.exists(): target_path = candidate.resolve() log.debug("Cross-module resolved: %s → %s", ref, target_path) break spec = self._load(target_path) ``` ### Technical Analysis The `$ref` value comes from the analyzed OpenAPI or Swagger document. Its file component is joined to the specification directory and canonicalized with `resolve()`, but the resulting path is never checked against ...[truncated 2089 chars]
Remediation
View remediation

T09 · Insecure Skill Coding Practices

Error
Location
api_case_gen.py:1650
Finding

Arbitrary YAML File Overwrite Through Unsanitized operationId

Content
View full analysis
list[Path]: analysis, cases_by_op = result out_dir = Path(output_dir) out_dir.mkdir(parents=True, exist_ok=True) written: list[Path] = [] for op_id, cases in cases_by_op.items(): if not cases: continue yaml_cases = [render_yaml_case(tc, self._config.env_mapping) for tc in cases] out_file = out_dir / f"{op_id}_cases.yaml" with open(out_file, "w", encoding="utf-8") as f: yaml.dump(yaml_cases, f, allow_unicode=True, default_flow_style=False, sort_keys=False) written.append(out_file) return written ``` The `op_id` key originates from the specification without validation: ```python operation_id=op.get("operationId", f"{method_name}_{path_key}"), ``` ### Technical Analysis An OpenAPI document controls `operationId`. The value is propagated into `cases_by_op` and then directly interpolated into an output filename. `Path(output_dir) / f"{op_id}_cases.yaml"` does not constrain the result to `output_dir`. An `operationId` containing path separators and parent-directory components can escape the selected output directory. The file is opened in `"w"` mode, which creates a new file or truncates an existing one. The hardcoded `_cases.yaml` suffix limits the most direct target names, but it does not prevent overwriting accessible files whose names end with that suffix. It may also place attacker-controlled generated YAML in unintended directories. ### Attack Path 1. An attacker creates a valid API specification containing a malicious operation identifier: ```yaml paths: /example: get: operationId: ../../shared/config ``` 2. Th ...[truncated 1224 chars]
Remediation
View remediation

T01 · Skill Instruction Hijacking

Warning
Location
api_case_gen.py:1780
Finding

Indirect Prompt Injection Through Untrusted API Descriptions

Content
View full analysis
str: """Build a structured prompt for LLM to supplement unextracted constraints.""" fields_to_review: list[dict[str, str]] = [] for analysis, _ in results: for op in analysis.operations: for f in CaseGenerator._collect_all_fields(op): if not f.unextracted_hints: continue fields_to_review.append({ "operation_id": op.operation_id, "method": op.method, "path": op.path, "field_path": f.path, "field_type": f.field_type, "required": str(f.required), "description": f.description, "hints": ", ".join(f.unextracted_hints), "existing_constraints": json.dumps( [{"kind": c["kind"], "value": c["value"]} for c in f.constraints], ensure_ascii=False, ), }) if not fields_to_review: return "No fields require AI review." prompt_parts: list[str] = [ "You are an API testing expert. For each field below, the regex-based", "constraint extractor could not fully parse the description. Analyze the", "original description and output supplementary test cases.\n", "## Instructions\n", "For each field, output a JSON object with:", "- `field_path`: the field's dot-separated path", "- `operation_id`: the operation ID", "- `missing_constraints`: array of {kind, value, reason}", "- `test_cases`: array of {id, priority, scenario, input, e ...[truncated 3639 chars]
Remediation
View remediation
Vulnerability Patterns
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
  • Memory PoisoningPersistent Context Injection, Context Window Stuffing, Memory Manipulation
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
Findings (8)

Memory Manipulation

High
Category
Memory Poisoning
Confidence
80% confidence
Finding

Skill manipulates agent memory, state, or stored context. Memory corruption can alter personality, override safety rules, or cause unpredictable behavior.

Content

Scanner excerpt · api_case_gen.py (reported line 878)May include surrounding context.

python
self._auth_emitted = False

    def reset(self) -> None:
        """Reset stateful counters (call before each independent analysis run)."""
        self._counter.clear()
        self._auth_emitted = False

Undeclared Tool Scope

Medium
Category
MCP Least Privilege
Confidence
93% confidence
Finding

The skill explicitly instructs the agent to read original YAML files and generate Markdown/YAML outputs, which implies file read/write behavior, yet it declares no permissions or allowed-tools scope. In a skill ecosystem, missing tool scoping increases the chance the skill is invoked with broader-than-necessary file access, enabling unintended access to repository contents or writes outside the expected output area.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The trigger conditions are very broad, covering common phrases like API test design, interface case generation, coverage analysis, and test matrix requests. That can cause the skill to auto-activate in many ordinary testing conversations, exposing users to file-reading and file-writing workflows they did not explicitly request and increasing the attack surface for prompt or context hijacking through unrelated API artifacts.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
84% confidence
Finding

The troubleshooting text treats non-Chinese descriptions as a cause of poor extraction, and later the limitations state that default description patterns target Chinese. This reflects a language-specific constraint without offering the user a language choice or opt-in, which can violate language/locale policy expectations for general-purpose skills.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

The line explicitly says default description patterns are oriented toward Chinese descriptions, while English APIs require regex extension. For a general-purpose skill, this imposes a language-specific default without presenting an opt-in language selection or documenting a narrowly justified regional scope.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

This Python skill embeds substantial fixed Chinese-language strings in generated test cases, markdown reports, coverage messages, and LLM supplement instructions, while offering no option for the user to choose language or locale. That creates a language/locale policy issue because the skill effectively forces one language for outputs across its main workflows.

Content

No source excerpt is available for this finding.

Unpinned Dependencies

Low
Category
Supply Chain
Confidence
96% confidence
Finding

The dependency is specified as pyyaml>=6.0 without an upper bound or exact pin, which makes builds non-reproducible and allows future releases to be pulled in without review. While this is primarily a supply-chain hygiene issue rather than an immediate exploit by itself, it can expose the skill to unexpected vulnerable or breaking versions over time.

Content

Scanner excerpt · requirements.txt (reported line 1)May include surrounding context.

text
pyyaml>=6.0

Unverifiable Dependency: pyyaml has 8 known advisory(ies) (CVE-2019-20477 (Deserialization of Untrusted Data in PyYAML); CVE-2020-1747 (Improper Input Validation in PyYAML); CVE-2020-14343 (Improper Input Validation in PyYAML) +5 more), but the manifest does not pin a version, so it is unknown whether the installed release is affected

Low
Category
Supply Chain
Confidence
88% confidence
Finding

PyYAML has a history of deserialization and input-validation advisories, and because the manifest does not pin a specific version, it is not possible to verify whether an affected release will be installed. In the context of an API test case generator, YAML may plausibly be used to parse OpenAPI or related configuration files, so use of an unsafe or outdated PyYAML version could matter if untrusted specs are processed.

Content

No source excerpt is available for this finding.