Back to skill

Security audit

problem-set-generator

Security checks for vulnerabilities and agentic risk

Overview

This looks like a simple contest problem generator, but it requests broad agent tools while its own metadata says those powers are not needed.

Review this before installing because the generator itself appears low-risk, but the skill manifest asks for broad tool access that is inconsistent with its own permission claims. Prefer a version that removes unused Write and WebFetch access, narrows Bash use to running the local CLI, and clarifies when the skill should activate.

Vulnerability Patterns
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
Vulnerability Patterns
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • Behavioral ASTexec() Call, eval() Call, Dynamic Import
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
Findings (12)

Intent-Code Divergence

High
Category
Not specified by scanner
Confidence
98% confidence
Finding

The manifest declares filesystem and subprocess permissions as false while simultaneously allowing Bash and file read/write tools, which directly imply subprocess and filesystem access. This is a high-risk integrity issue because policy engines, reviewers, or sandboxing logic may trust the permission block and under-enforce controls, enabling unexpected local execution or file manipulation.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
92% confidence
Finding

The English section's example output uses Chinese-only values such as "入门" and "待审(组委会人工审核后方可发布)", and the README does not state that users can select output language or locale. This is a natural-language locale policy concern because the skill appears to impose a specific language in generated/output text without explicit opt-in or documented justification.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
88% confidence
Finding

In the English README, the example warnings array contains Chinese-only text, but no accompanying statement says warnings are intentionally Chinese for a region-specific audience or that users may choose another language. This can violate language/locale policy by forcing a specific language in user-facing messaging.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

The manifest says the skill should be used '当需要赛事赛题生成器能力时' and lists broad trigger terms such as '赛事' and '判定器'. These terms are common within contest-related discussions and do not clearly distinguish when the skill should activate versus when it should not, increasing the risk of unintended invocation.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Medium
Category
Not specified by scanner
Confidence
92% confidence
Finding

The documentation claims the skill requires no local scripts, files, or code-execution environment, yet the manifest allows Bash plus file read/write tools. This mismatch can mislead operators and automated policy gates into treating the skill as lower risk than it really is, increasing the chance that code execution or file access is granted without appropriate scrutiny.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

The Chinese usage text repeats the same unsafe discrepancy by stating no local script or file operations are needed while the skill declares tools and rationale consistent with local execution and file access. In multilingual skills, inconsistent claims are especially dangerous because reviewers may rely on only one language version and miss the true privilege level.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
90% confidence
Finding

The file’s user-facing docstrings, CLI descriptions, argument help text, errors, warnings, notes, and generated content are all written in Chinese, with no option to select another language or statement limiting the tool to a justified Chinese-only context. This creates a natural-language locale constraint that is imposed by default rather than offered as a user choice.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
84% confidence
Finding

The English usage text states that output_format is 'forcing JSON IR return', and the Chinese usage section similarly says it '强制返回 JSON IR'. This imposes a fixed output convention rather than offering user choice or documenting opt-in, which can conflict with language/locale policy expectations around user-selectable output behavior.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
80% confidence
Finding

The workflow says to 'ensure output_format points to JSON IR,' which prescribes a single output convention for all uses. Because the documentation does not offer an opt-in or alternative, this reads as a hard requirement rather than a user-selected preference.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
80% confidence
Finding

The Chinese workflow instructs users to ensure output_format points to JSON IR format, mirroring the same hard requirement as the English section. This creates a fixed output convention without documenting user choice or a clearly justified exception.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
91% confidence
Finding

The file content is written entirely in Chinese and does not offer any language or locale choice for users or agents. Under the stated policy, forcing a specific language without opt-in can be a natural-language policy violation unless the locale constraint is explicitly justified.

Content

No source excerpt is available for this finding.

Dynamic attribute access via getattr()

Low
Category
Dangerous Code Execution
Confidence
50% confidence
Finding

Dynamic getattr() with a non-literal attribute name can access arbitrary object attributes, potentially bypassing access controls.

Content

Scanner excerpt · problem-set-generator.py (reported line 94)May include surrounding context.

python
out_list.append({"tool": META["slug"], "input": d, "errors": [str(e)], "rc": 2, "aigc_mark": AIGC})
        print(json.dumps(out_list, ensure_ascii=False, indent=2))
        sys.exit(0 if allok else 2)
    args = {k: getattr(ns, k) for k in argnames}
    try:
        res, rc = _run(args)
    except GateError as e: