Back to skill

Security audit

testcase-generator

Security checks for vulnerabilities and agentic risk

Overview

This test-generation skill is coherent, but it can read and save real database and cache data without enough safeguards, so it needs review before use.

Install only if you are comfortable giving the skill access to the referenced application config, MySQL database, Redis cache, and generated output files. Use non-production, read-only credentials; prefer schema-only or sanitized test data; restrict tables and Redis prefixes tightly; avoid broad patterns; review outputs for secrets or personal data before sharing them with an LLM or another test-execution skill.

Vulnerability Patterns
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Findings (2)

T09 · Insecure Skill Coding Practices

Error
Location
scripts/data_reader.py:224
Finding

SQL Injection Through Unvalidated Table Identifiers

Content
View full analysis
List[Dict]: """ Execute a custom SQL query. Args: sql: SQL statement Returns: Query result list """ if not self.connection: if not self.connect(): return [] try: with self.connection.cursor() as cursor: cursor.execute(sql) return cursor.fetchall() except Exception as e: print(f"SQL execution failed: {e}") return [] ``` ### Technical Analysis The `table_name` argument originates from the command-line `--tables` option and is directly interpolated into SQL statements. It is not checked against a table allowlist, validated as a single SQL identifier, or safely quoted. Prepared-statement placeholders generally cannot be used for table names. Consequently, dynamic identifiers must be validated separately. The current implementation permits a crafted table argument to introduce SQL syntax into the `SELECT` statement. For example, an attacker could attempt to inject clauses or a `UNION` expression, subject to column compatibility and database-driver restrictions. The `where` argument is also concatenated directly. Although the current CLI does not expose it, this becomes another injection sink if the class is imported and called by other Skill components. Similarly, `execute ...[truncated 1799 chars]
Remediation
View remediation
str: if not IDENTIFIER_PATTERN.fullmatch(value): raise ValueError(f"Invalid SQL identifier: {value!r}") return value ``` 2. Enforce an explicit allowlist of schemas and tables approved for test-data access. Syntax validation alone does not prevent callers from selecting sensitive but valid table names. 3. Quote validated identifiers with a database-aware mechanism. Do not treat quoting as a substitute for validation and authorization. 4. Remove the `where` argument if it is not required. If filtering is needed, expose structured column/operator/value inputs, validate columns and operators, and bind values using query parameters. 5. Remove `execute_query()` unless arbitrary SQL is essential. If it must remain, restrict it to trusted internal callers and enforce read-only statements rather than accepting arbitrary text. 6. Use a dedicated read-only MySQL account that can access only approved test schemas and tables. Do not use production application credentials. 7. Add security tests covering spaces, comments, quotes, semicolons, `UNION`, qualified identifiers, and attempts to access non-allowlisted tables. ]]>

T05 · Unauthorized Access and Privilege Escalation

Warning
Location
scripts/data_reader.py:407
Finding

Over-Privileged Collection and Unredacted Disclosure of Database and Cache Data

Content
View full analysis
= limit: break if cursor == 0: break return result ``` Complete MySQL records and Redis values are retained and then written or printed without redaction: ```python result = { 'timestamp': datetime.now().isoformat(), 'config': args.config, 'mysql_data': {}, 'redis_data': {} } if args.tables: for table in args.tables: print(f"\nReading MySQL table: {table}") table_data = reader.read_mysql_table(table, limit=args.limit) if args.schema_only: print(f" - Field count: {len(table_data['schema'])}") for field in table_data['schema']: print(f" - {field['Field']}: {field['Type']}") else: print(f" - Record count: {table_data['count']}") result['mysql_data'][table] = table_data if args.redis_patterns: for pattern in args.redis_patterns: print(f"\nReading Redis pattern: {pattern}") redis_data = reader.read_redis_pattern(pattern) print(f" - Key count: {len(redis_data)}") result['redis_data'][pattern] = redis_data reader.close() if args.output: with open(args.output, 'w', encoding='utf-8') as f: json.dump(result, f, ensure_ascii=False, indent=2) print(f"\nData saved to: {args.output}") else: ...[truncated 3756 chars]
Remediation
View remediation
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
Findings (11)

Tp4

High
Category
MCP Tool Poisoning
Confidence
97% confidence
Finding

The stated purpose is test-case generation, but the documented behavior includes direct MySQL/Redis access, reading real production-like data, and acting as a generic data extraction component. This mismatch is dangerous because users may authorize the skill for a narrow task while it exercises broader, more sensitive data-access capabilities than disclosed.

Content

No source excerpt is available for this finding.

Undeclared Tool Scope

Medium
Category
MCP Least Privilege
Confidence
95% confidence
Finding

The skill declares no explicit tool/permission scope even though it is designed to read configuration files, access environment-derived configuration, and generate output files. In an agent setting, missing scope declarations increases the chance of overbroad file and data access beyond what a user would reasonably infer from the manifest.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

The skill is built around reading real MySQL/Redis data and configuration-derived credentials, yet it provides no prominent warning about sensitive-data exposure, least-privilege expectations, or privacy impact. In this context, the model may ingest confidential records and secrets into prompts or outputs, creating unnecessary data leakage risk.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

Medium
Category
Not specified by scanner
Confidence
90% confidence
Finding

The workflow says the skill continues into invoking another skill to execute generated tests, which exceeds a generation-only capability boundary. Hidden cross-skill delegation is risky because it can transform a seemingly passive content-generation skill into an action-taking workflow with broader operational effects.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The instructions tell users to run a data-reading script against application configuration, tables, and Redis keys without warning that this may extract credentials and sensitive business data from local configs and backing stores. This is especially risky because the workflow then feeds retrieved data into LLM-based test generation, increasing the blast radius of any exposed secrets or private records.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

The usage instructions explicitly direct the agent to call another skill to execute test cases after generation, despite the manifest positioning this as a generator. That scope expansion is security-relevant because users may not expect or approve downstream execution capabilities when enabling a data-analysis skill.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

The config loader resolves arbitrary ${ENV:default} placeholders from process environment variables, allowing any YAML value to pull in secrets such as cloud credentials, tokens, or database passwords. In a skill whose stated purpose is generating test cases from MySQL/Redis data, broad environment-variable access is an unnecessary credential-access capability and expands the blast radius if an attacker can influence the config file.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

Medium
Category
Not specified by scanner
Confidence
88% confidence
Finding

The script serializes all collected MySQL and Redis contents to a user-specified JSON path, enabling bulk export of potentially sensitive records beyond the narrow skill description of test-case generation. This creates a straightforward exfiltration path to local files or mounted directories, especially risky in agent/tooling environments where downstream components may upload or retain artifacts.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

Writing raw database and Redis contents to a user-controlled JSON file without warning, masking, or access controls can expose credentials, personal data, session tokens, and other sensitive material. Because the tool reads broad table/key contents and then emits them verbatim, accidental disclosure is highly plausible even without a sophisticated attacker.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
96% confidence
Finding

The main instructional content begins in Chinese and continues primarily in that language, but the file does not indicate that the skill is region-specific or offer the user a language option. This creates a natural-language locale policy concern because the skill effectively forces one language without opt-in.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
88% confidence
Finding

The file's docstrings, help text, and runtime messages are presented in Chinese, which effectively fixes the interaction language for users of the skill. There is no indication that this locale restriction is optional, user-selected, or justified as a region-specific tool.

Content

No source excerpt is available for this finding.

Static analysis

Detected: suspicious.exposed_secret_literal

File appears to expose a hardcoded API secret or token.

Critical
Code
suspicious.exposed_secret_literal
Location
scripts/data_reader.py:150

File appears to expose a hardcoded API secret or token.

Critical
Code
suspicious.exposed_secret_literal
Location
SKILL.md:201