Back to skill

Security audit

ai-security-redteam

Security checks for vulnerabilities and agentic risk

Overview

This is a local AI red-team reference and CLI helper that is mostly defensive and disclosed, with some documentation and trigger-scope cautions but no evidence of hidden execution, networking, persistence, or data exfiltration.

Install only for authorized AI security assessment work. Treat its generated prompts and red-team cases as defensive test material, avoid using them against systems without written permission, and be aware the package is primarily Chinese and its broad trigger words may cause it to load for general security discussions.

Vulnerability Patterns
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
Vulnerability Patterns
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • Behavioral ASTexec() Call, eval() Call, Dynamic Import
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
Findings (27)

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
92% confidence
Finding

The file is written entirely in Chinese and does not indicate that other languages are supported or that Chinese is required for a region-specific or compliance-specific reason. This creates a natural-language policy concern because it imposes a locale choice on users and agents without opt-in.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

The README lists activation terms such as “OWASP”, “渗透测试”, and “安全测试” without narrowing context or providing exclusion conditions. These phrases are broad enough to match many ordinary security discussions, making the skill's invocation scope ambiguous.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

This markdown file presents all user-facing content in Chinese, including the title, findings, and usage guidance, with no indication that the language is optional or that the skill is intended only for a Chinese-speaking audience. Under the policy criteria, forcing a specific language without user opt-in is a natural-language policy violation.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Medium
Category
Not specified by scanner
Confidence
89% confidence
Finding

Line L30 states there are no informational reminders, yet lines L68-L72 include advisory recommendations, including a note that regulatory reference content changes over time and should be updated. This is a direct contradiction within the document's own intent statements, not merely omitted detail.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
85% confidence
Finding

The skill advertises broad trigger terms such as AI安全, 红队测试, 提示注入, and 渗透测试 in metadata/description, which can cause the skill to activate in a wide range of general security conversations rather than only when the user explicitly wants this specific red-team toolkit. In an agent ecosystem, over-broad activation can unnecessarily route sensitive prompts into a high-risk security-focused skill, increasing the chance of unsafe assistance, scope confusion, or policy-boundary bypass attempts.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
90% confidence
Finding

The trigger-scenario table maps broad keywords like AI安全, 攻击面, 红队测试, 提示注入, and 越权 directly to chapter loading, but does not impose strict gating for when the skill should stay inactive. Although the document contains safety disclaimers elsewhere, the activation logic itself remains permissive, which can lead to over-triggering in ambiguous contexts and expose detailed red-team workflows where only high-level defensive guidance was appropriate.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
92% confidence
Finding

The trigger list contains very broad terms such as AI security, OWASP, penetration testing, and security testing, which can match many ordinary user requests and cause this skill to activate outside its intended scope. In an agent ecosystem, over-broad auto-routing can expose users to offensive-security-oriented content or workflows when they only wanted general guidance, increasing the chance of unsafe or mis-scoped assistance.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
86% confidence
Finding

This markdown file is entirely written in Chinese and presents the skill's purpose and usage only in that language. Under the policy, forcing a specific language without user opt-in can be a natural-language policy violation unless the locale restriction is explicitly justified, which is not stated here.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The module docstring describes the verifier as '本地闭环、零网络' and later the program note says '零数据采集', which implies no data collection activity. However, check_security walks the entire project tree and opens .md, .py, .json, .svg, and .txt files to inspect their contents, which is a form of local data collection and analysis.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

This code contains natural-language text that sets the skill context entirely in Chinese, which can indicate a locale-specific behavior without user opt-in. The policy requires flagging language or locale constraints when the skill does not offer a choice or clearly justify the restriction.

Content

No source excerpt is available for this finding.

subprocess module call

Medium
Category
Dangerous Code Execution
Confidence
70% confidence
Finding

subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Content

Scanner excerpt · verify/ai_redteam_quality_test.py (reported line 52)May include surrounding context.

python
def check_tools():
    items = []
    try:
        r = subprocess.run([PYTHON, TOOL, "risk", "--system", "客服AI,接知识库,可调用CRM和发邮件"],
                           capture_output=True, text=True, timeout=60)
        out = r.stdout or ""
        items.append({"name": "risk 命令执行", "pass": r.returncode == 0 and "injection" in out and "overreach" in out})

subprocess module call

Medium
Category
Dangerous Code Execution
Confidence
70% confidence
Finding

subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Content

Scanner excerpt · verify/ai_redteam_quality_test.py (reported line 56)May include surrounding context.

python
capture_output=True, text=True, timeout=60)
        out = r.stdout or ""
        items.append({"name": "risk 命令执行", "pass": r.returncode == 0 and "injection" in out and "overreach" in out})
        r2 = subprocess.run([PYTHON, TOOL, "cases", "--surface", "injection"], capture_output=True, text=True, timeout=60)
        items.append({"name": "cases 命令执行", "pass": r2.returncode == 0 and "注入" in (r2.stdout or "")})
        r3 = subprocess.run([PYTHON, TOOL, "grade", "--desc", "攻击者可注入指令让AI泄露全部客户数据"],
                            capture_output=True, text=True, timeout=60)

subprocess module call

Medium
Category
Dangerous Code Execution
Confidence
70% confidence
Finding

subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Content

Scanner excerpt · verify/ai_redteam_quality_test.py (reported line 58)May include surrounding context.

python
items.append({"name": "risk 命令执行", "pass": r.returncode == 0 and "injection" in out and "overreach" in out})
        r2 = subprocess.run([PYTHON, TOOL, "cases", "--surface", "injection"], capture_output=True, text=True, timeout=60)
        items.append({"name": "cases 命令执行", "pass": r2.returncode == 0 and "注入" in (r2.stdout or "")})
        r3 = subprocess.run([PYTHON, TOOL, "grade", "--desc", "攻击者可注入指令让AI泄露全部客户数据"],
                            capture_output=True, text=True, timeout=60)
        items.append({"name": "grade 命令执行", "pass": r3.returncode == 0 and "严重" in (r3.stdout or "")})
        r4 = subprocess.run([PYTHON, TOOL, "report"], capture_output=True, text=True, timeout=60)

subprocess module call

Medium
Category
Dangerous Code Execution
Confidence
70% confidence
Finding

subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Content

Scanner excerpt · verify/ai_redteam_quality_test.py (reported line 61)May include surrounding context.

python
r3 = subprocess.run([PYTHON, TOOL, "grade", "--desc", "攻击者可注入指令让AI泄露全部客户数据"],
                            capture_output=True, text=True, timeout=60)
        items.append({"name": "grade 命令执行", "pass": r3.returncode == 0 and "严重" in (r3.stdout or "")})
        r4 = subprocess.run([PYTHON, TOOL, "report"], capture_output=True, text=True, timeout=60)
        items.append({"name": "report 命令执行", "pass": r4.returncode == 0 and "红队测试报告" in (r4.stdout or "")})
        r5 = subprocess.run([PYTHON, TOOL, "fix", "--vuln", "injection"], capture_output=True, text=True, timeout=60)
        items.append({"name": "fix 命令执行", "pass": r5.returncode == 0 and "隔离" in (r5.stdout or "")})

subprocess module call

Medium
Category
Dangerous Code Execution
Confidence
70% confidence
Finding

subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Content

Scanner excerpt · verify/ai_redteam_quality_test.py (reported line 63)May include surrounding context.

python
items.append({"name": "grade 命令执行", "pass": r3.returncode == 0 and "严重" in (r3.stdout or "")})
        r4 = subprocess.run([PYTHON, TOOL, "report"], capture_output=True, text=True, timeout=60)
        items.append({"name": "report 命令执行", "pass": r4.returncode == 0 and "红队测试报告" in (r4.stdout or "")})
        r5 = subprocess.run([PYTHON, TOOL, "fix", "--vuln", "injection"], capture_output=True, text=True, timeout=60)
        items.append({"name": "fix 命令执行", "pass": r5.returncode == 0 and "隔离" in (r5.stdout or "")})
    except Exception as e:
        items = [{"name": f"执行失败: {e}", "pass": False}]

subprocess module call

Medium
Category
Dangerous Code Execution
Confidence
70% confidence
Finding

subprocess module calls execute external commands. Without careful input validation, this enables command injection.

Content

Scanner excerpt · verify/ai_redteam_quality_test.py (reported line 79)May include surrounding context.

python
items = []
    for desc, expects in cases:
        try:
            r = subprocess.run([PYTHON, TOOL, "grade", "--desc", desc], capture_output=True, text=True, timeout=60)
            ok = r.returncode == 0 and any(e in (r.stdout or "") for e in expects)
            items.append({"name": f"grade「{desc[:16]}…」→{expects[0]}", "pass": ok})
        except Exception as e:

Intent-Code Divergence

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

The note in the results object states '零数据采集', yet several checks read SKILL.md, multiple reference markdown files, and the tool source, and process those contents into pass/fail outputs. That is not merely incomplete documentation; it directly conflicts with the stated no-data-collection claim.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
84% confidence
Finding

The generated results include a fixed Chinese-language note, which is a natural-language locale constraint embedded in the code. Because no alternative language choice or explicit justification is present in this file, it may violate the language/locale policy.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
89% confidence
Finding

This JSON file includes user-visible natural-language text in Chinese (for example the note field), and the rest of the file continues with Chinese labels and check names. Under the policy, forcing a specific language without user opt-in can be a language/locale policy violation, and this file does not indicate any optional language selection or region-specific justification.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
77% confidence
Finding

The main description and trigger guidance are presented primarily in Chinese, which can imply a language default for activation and use. While English appears later in the file, the top-level skill description does not explicitly state that users may choose their preferred language.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Low
Category
Not specified by scanner
Confidence
72% confidence
Finding

L70 characterizes the package as zero-network and purely local, but L72 says the regulatory quick-reference content depends on continuously changing external regulations and should be updated over time. While not proving runtime network access, the documentation's stated closed/local nature conflicts with acknowledging externally dependent content that requires ongoing synchronization.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
95% confidence
Finding

L3 presents the skill description and invocation guidance entirely in Chinese, with no indication that users may choose another language or locale. This can constitute a language-policy issue when the skill appears to impose a specific language by default rather than offering an explicit choice.

Content

No source excerpt is available for this finding.

Vague Triggers

Low
Category
Not specified by scanner
Confidence
88% confidence
Finding

This markdown file states the skill is used to answer questions like 'how to classify vulnerabilities', 'repair suggestions', and 'retest process', but it does not define exact invocation phrases, boundaries, or exclusion conditions. Because the described questions are broad and common within security discussions, the activation scope may be ambiguous and could lead to unintended invocation.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Low
Category
Not specified by scanner
Confidence
78% confidence
Finding

Lines L15-L16 assert a concrete behavioral guarantee about ai_redteam_toolkit.py ('不发起攻击、不联网'). In this file, that is an intent-level claim about another component's runtime behavior, but there is no code here to verify it; if the referenced tool behaved differently, this documentation would directly contradict the implementation.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
91% confidence
Finding

The module docstring and later user-facing console messages indicate the skill is designed for a single language/locale. Under the policy, forcing a specific language without user opt-in can be a natural-language policy violation unless the locale restriction is clearly documented and justified.

Content

No source excerpt is available for this finding.