T09 · Insecure Skill Coding Practices
Error
- Location
- SKILL.md:102
- Finding
- Untrusted Test Suites and Notebooks Are Executed Without Isolation## Vulnerability Details **File Location**: `SKILL.md`, lines 102–107 **Vulnerability Type**: Unsafe execution of repository-controlled code **Risk Level**: High **Vulnerable code:** ```markdown ### 5. Proof of Work ```bash pytest tests/math/ --benchmark jupyter nbconvert --execute derivation.ipynb ``` **Verification:** Run `pytest -v tests/math/` to verify. ``` ### Technical Analysis The skill instructs the agent to execute test files and a Jupyter notebook from the project under review without requiring a prior trust assessment, source inspection, sandbox, privilege reduction, or network restriction. Both operations execute repository-controlled Python code: - `pytest` imports test modules during collection and executes module-level statements, fixtures, hooks, plugins, and test bodies. - `jupyter nbconvert --execute` runs every executable cell in the specified notebook. - Repeating execution through the verification command does not provide an independent safety check; it executes the same untrusted test surface again. A malicious repository can therefore place arbitrary Python instructions in `tests/math/`, pytest configuration or hooks, or `derivation.ipynb`. When an agent follows the workflow, those instructions run with the permissions and environment of the agent process. The fixed relative paths do not make the commands safe because their contents are controlled by the repository being audited. ### Attack Path 1. An attacker supplies a repository containing an apparently legitimate mathematical implementation. 2. The attacker adds malicious module-level code, fixtures, or hooks under `tests/math/`, or malicious executable cells to `derivation.ipynb`. 3. A reviewer loads this skill and follows its “Proof of Work” instructions. 4. The reviewer executes `pytest tests/math/ --benchmark`, `pytest -v tests/math/`, or `jupyter nbconvert --execute derivation.ipynb`. 5. The attacker-controlled Python cod ...[truncated 1000 chars]
- Remediation
- ## Remediation Suggestions 1. Treat all repository tests, pytest configuration, plugins, hooks, and notebooks as untrusted executable code. 2. Require explicit inspection and user approval before executing project-controlled files. Static review must occur before test collection or notebook execution. 3. Run verification only in a disposable sandbox or ephemeral container with: - No host filesystem mounts except a read-only project copy and a dedicated writable output directory. - No inherited secrets, SSH agents, cloud credentials, API tokens, or sensitive environment variables. - Network access disabled by default. - A non-root user, dropped Linux capabilities, resource limits, and process/time limits. - No access to host container sockets or privileged device interfaces. 4. Disable untrusted third-party pytest plugin autoloading, for example by setting `PYTEST_DISABLE_PLUGIN_AUTOLOAD=1`, and explicitly allow only reviewed plugins. 5. Do not execute notebooks by default. Move notebook execution behind a separate, clearly labeled opt-in trust gate. 6. Replace the current unconditional workflow with hardened guidance, such as: ```markdown Before running project-controlled tests or notebooks, inspect them and obtain explicit approval. Execute them only inside a disposable, non-root sandbox with no secrets, no host filesystem access, and networking disabled. ``` 7. Record the sandbox configuration and exact reviewed commit alongside test evidence so results remain attributable and reproducible.