Back to skill

Security audit

software_test_case_design

Security checks for vulnerabilities and agentic risk

Overview

This is a markdown-only test-case design skill whose sensitive topics are framed as QA test scenarios, not as hidden execution or data-access behavior.

Installers should treat this as a test-case-writing aid, not an execution harness. Review generated cases before running them, replace example passwords and bearer tokens with test-only placeholders, avoid production data or payment rails, and narrow prompts when asking for AI-agent tests because the "Agent" trigger is broad.

Vulnerability Patterns
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
Vulnerability Patterns
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
Findings (14)

Vague Triggers

High
Category
Not specified by scanner
Confidence
98% confidence
Finding

Using the standalone keyword 'Agent' as a trigger is highly ambiguous because it appears in many unrelated requests, including references to the assistant itself or generic business agents. This can misroute a wide range of prompts into agent-testing behavior, causing inappropriate file loads and instruction application in contexts the user did not intend.

Content

No source excerpt is available for this finding.

Vague Triggers

High
Category
Not specified by scanner
Confidence
97% confidence
Finding

The instruction mapping table repeats the ambiguous 'Agent' trigger, reinforcing the same unsafe routing logic in a second authoritative section of the skill. Duplicating the ambiguity increases the chance that implementations or maintainers preserve the overbroad behavior and apply the agent-testing path to unrelated prompts.

Content

No source excerpt is available for this finding.

Credential Access

High
Category
Privilege Escalation
Confidence
70% confidence
Finding

Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Content

Scanner excerpt · references/checklists/mobile-app-checklist.md (reported line 141)May include surrounding context.

md
- [ ] Important information has haptic feedback

## Storage & Data (10 items)
- [ ] SharedPreferences/Keychain read/write works correctly
- [ ] SQLite/Room/CoreData database operations work correctly
- [ ] Image caching strategy takes effect
- [ ] API data caching strategy takes effect

YARA rule 'agent_skill_prompt_injection_hidden_instructions': Prompt injection or hidden instructions embedded in AI agent skill text [agent_skills]

High
Category
YARA Match
Confidence
80% confidence
Finding

YARA rule matched a hack tool or exploit indicator (offensive tools, reconnaissance, privilege escalation, or exploit frameworks).

Content

Scanner excerpt · references/core-capabilities/agent-testing.md (reported line 1)May include surrounding context.

md
# AI Agent Testing

> This document defines specialized testing dimensions and strategy highlights for AI Agents (intelligent agents). Agents are cross-platform capabilities that can be embedded in Apps, Mini-Programs, H5, Desktop, PC Web, and other hosts, and are tested in conjunction with host platform testing.
> See `references/examples/format-spec.md` for output format and `references/checklists/agent-checklist.md` for the checklist.

---

## I. Task Completion

### Focus Areas
- User intent understanding accuracy: explicit instructions, implicit requirements, ambiguous requirements, intent

Instruction Override

High
Category
Prompt Injection
Confidence
90% confidence
Finding

This pattern attempts to override system instructions or ignore safety constraints. Without LLM analysis, manual review is recommended.

Content

Scanner excerpt · references/core-capabilities/agent-testing.md (reported line 66)May include surrounding context.

md
- Resource exhaustion handling: graceful degradation when memory/Token limits are reached

### Common Defects
- Being induced to bypass security restrictions and perform dangerous operations
- Leaking API Keys or internal information in logs/output
- Having role hijacked by user prompts
- Failing to decline clearly harmful requests

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

The skill description is very broad and can match many ordinary test-related requests without clear exclusions or activation boundaries. Overbroad routing can cause this skill to be invoked in unintended contexts, leading to irrelevant instructions being loaded and potentially displacing a more appropriate or safer skill.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
88% confidence
Finding

This markdown file describes testing of authorizations, local storage, location, sharing, messaging, and payments, but it does not include any warning about potential effects on user privacy, stored data, or financial actions. For markdown files, omissions of warnings about behaviors that could affect user data, privacy, or system integrity should be flagged.

Content

No source excerpt is available for this finding.

Autonomous Decision Making

Medium
Category
Excessive Agency
Confidence
75% confidence
Finding

Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.

Content

Scanner excerpt · references/core-capabilities/agent-testing.md (reported line 70)May include surrounding context.

md
- Leaking API Keys or internal information in logs/output
- Having role hijacked by user prompts
- Failing to decline clearly harmful requests
- Executing high-risk operations without confirmation
- Session data leakage when one instance serves multiple users
- Improper handling of extreme inputs (empty/overly long/special characters) leading to exceptions
- No reasonable degradation on service exceptions, resulting in direct failure

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
84% confidence
Finding

This markdown file includes example account credentials in a precondition and later shows an Authorization Bearer token in API test steps. Under the markdown variant of SQP-2, the document should warn users not to use real credentials or production secrets when following examples that affect privacy or system integrity.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
83% confidence
Finding

The checklist includes a mandatory item for 'Chinese input method works correctly,' which imposes a specific language/locale requirement in natural language. Because the document does not state that it is intended for a China-specific product or offer locale scoping, this can be read as an unjustified language policy constraint.

Content

No source excerpt is available for this finding.

Missing User Warnings

Low
Category
Not specified by scanner
Confidence
81% confidence
Finding

The AI Agent testing example instructs creation of a file named test.txt, which is a write operation affecting system state. The markdown description does not include any warning that such steps should be confined to a safe test workspace or may modify existing files.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
82% confidence
Finding

The testing guidance hard-codes shortcut notation like "Ctrl+C/V/Z/Y/A/S/F" and later references English UI text such as "Open with XX". In a natural-language/platform guidance document, this can amount to an implicit language/locale assumption without noting localization or user-language variation.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
78% confidence
Finding

The guidance explicitly requires testing Chinese input method composition events, which imposes a language/locale-specific requirement in a general mobile web testing document. The file does not state that this checklist is limited to Chinese-language products or markets, nor does it offer locale-based applicability guidance.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
87% confidence
Finding

The example says to paste a '10,000-character Chinese text,' which imposes a specific language in natural-language guidance. Under the policy, forcing a language or locale without user opt-in or a documented justification is a violation.

Content

No source excerpt is available for this finding.

Static analysis

Detected: suspicious.exposed_secret_literal

File appears to expose a hardcoded API secret or token.

Critical
Code
suspicious.exposed_secret_literal
Location
references/examples/format-spec.md:104