Security audit

Bo Ren

Security checks for vulnerabilities and agentic risk

Overview

This notebook-building skill mostly does what it claims, but its quiz-generation code can turn untrusted quiz text into executable Python inside generated notebooks.

Review generated notebooks before running quiz cells, especially when quiz questions, hints, or IDs come from untrusted sources. Use the skill only on intended project files, specify output paths clearly, and do not rely on its local hash quiz system for secure exams or certification grading.

A.I.G

Vulnerability Patterns

Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally

Findings (3)

T09 · Insecure Skill Coding Practices

Error

Location: scripts/nb_helpers.py:348
Finding: Arbitrary Python Code Injection Through Quiz Cell Generation
Content: View full analysis
Remediation: View remediation

T09 · Insecure Skill Coding Practices

Error

Location: scripts/nb_helpers.py:417
Finding: Arbitrary Python Code Injection Through Quiz Summary Generation
Content: View full analysis
Remediation: View remediation

T09 · Insecure Skill Coding Practices

Warning

Location: scripts/nb_helpers.py:303
Finding: Quiz Answers Are Recoverable Due to a Public Fixed Salt and Truncated Fast Hash
Content: View full analysis
str: """Hash an answer with a salt.""" if isinstance(answer, (list, tuple)): normalized = str(tuple(answer)) elif isinstance(answer, bool): normalized = str(answer) elif isinstance(answer, (int, float)): normalized = str(answer) elif isinstance(answer, str): normalized = answer.strip().lower() else: normalized = str(answer) raw = f"{_HASH_SALT}:{normalized}" return hashlib.sha256(raw.encode("utf-8")).hexdigest()[:16] ``` The same salt and truncated comparison are embedded in every generated quiz cell: ```python def _check_{question_id}(user_ans): _salt = "{_HASH_SALT}" if isinstance(user_ans, (list, tuple)): _n = str(tuple(user_ans)) elif isinstance(user_ans, bool): _n = str(user_ans) elif isinstance(user_ans, (int, float)): _n = str(user_ans) elif isinstance(user_ans, str): _n = user_ans.strip().lower() else: _n = str(user_ans) _h = _hl.sha256(f"{{_salt}}:{{_n}}".encode()).hexdigest()[:16] if _h == "{answer_hash}": ``` ### Technical Analysis The mechanism is described as hiding answers, but it does not provide meaningful confidentiality against offline guessing: - The salt is a constant: `nb_builder_quiz_v1`. - The salt is embedded in the generated notebook and is therefore public. - SHA-256 is deliberately fast, allowing large numbers of guesses per second. - Only the first 16 hexadecimal characters are retained, reducing the comparison to 64 bits. - Typical quiz answers have extremely low entropy, such as booleans, small integers, short strings, tuples, or multiple-choice values. - The exact normalization algorithm is inclu ...[truncated 1626 chars]
Remediation: View remediation

SkillSpector

Vulnerability Patterns

Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access

Findings (5)

Undeclared Tool Scope

Medium

Category: MCP Least Privilege
Confidence: 91% confidence
Finding: The skill clearly enables local file reads and writes by instructing the agent to load, modify, merge, and save notebooks, but it does not declare any explicit tool scope or permission boundaries. That creates an authorization gap where an agent may perform filesystem operations beyond what a reviewer or platform policy expects, increasing the risk of unintended file modification.
Content: No source excerpt is available for this finding.

Missing User Warnings

Medium

Category: Not specified by scanner
Confidence: 95% confidence
Finding: The skill repeatedly instructs the agent to execute Python via a terminal and save notebooks, while framing this as mandatory behavior, but it does not prominently warn that these actions will create or overwrite local files. This can lead to silent filesystem changes and makes it easier for a prompt-injected or mistaken workflow to modify user data without sufficiently informed consent.
Content: No source excerpt is available for this finding.

Skill Enumeration

Medium

Category: Agent Snooping
Confidence: 80% confidence
Finding: Skill enumerates or reads other installed skills. Access to other skills' SKILL.md files or the skills directory reveals prompt instructions, capabilities, and secrets that should be invisible to peer skills.
Content: Scanner excerpt · SKILL.md (reported line 40)May include surrounding context.
md
1. 此 SKILL.md 文件所在目录的同级 `scripts/` 子目录 2. 即 SKILL.md 的父目录 + `/scripts` 例如如果此 SKILL.md 位于 `/home/user/.codebuddy/skills/notebook-builder/SKILL.md`，则 `<技能脚本目录>` 应替换为 `/home/user/.codebuddy/skills/notebook-builder/scripts`。 ## 辅助脚本

Natural-Language Policy Violations

Medium

Category: Not specified by scanner
Confidence: 96% confidence
Finding: This Python file contains user-facing natural-language documentation and command-line messages exclusively in Chinese, starting with the module docstring and continuing throughout the file. Because the skill does not offer an opt-in language choice or document a justified locale restriction, it conflicts with the language/locale policy for natural-language content.
Content: No source excerpt is available for this finding.

Natural-Language Policy Violations

Low

Category: Not specified by scanner
Confidence: 97% confidence
Finding: The README uses Chinese throughout and does not indicate that users may choose another language or locale. Under the policy rule, forcing a specific language without user opt-in is a natural-language policy concern.
Content: No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.