Security audit
Security checks for vulnerabilities and agentic risk
This is a real spec-writing pipeline, but it gives installer and model-driven phases broad code execution and repository mutation authority without enough containment.
Review before installing. Avoid the one-line curl | bash path; install only from pinned, reviewed commits. Treat briefs, repository files, generated spec.md, and provider outputs as untrusted. Run provider CLIs in a sandbox or disposable worktree, require safe permission modes, and do not merge results unless the final diff is limited to expected files such as spec.md.
SKILL.md:20Mutable Remote Installer Is Piped Directly into Bash
scripts/install.sh:14Unpinned Repositories Are Downloaded and Immediately Imported
scripts/adversarial_spec.py:617Model-Authored Acceptance Directives Are Executed Without Local Security Validation
scripts/phases/phase_write.py:47Agent Can Modify Arbitrary Repository Files That Are Then Committed and Merged
SKILL.md:170Recommended Provider Configuration Disables Permission Checks for Untrusted Input
The manifest describes a spec-writing skill, but the document also includes installation, bootstrap, repository manipulation, and remote-fetch behavior not reflected in the declared purpose. This mismatch hides materially risky behavior from reviewers and policy engines, which can lead to under-scoped approvals for a skill that performs broader system actions.
The manifest describes a spec-writing skill, but the document also includes installation, bootstrap, repository manipulation, and remote-fetch behavior not reflected in the declared purpose. This mismatch hides materially risky behavior from reviewers and policy engines, which can lead to under-scoped approvals for a skill that performs broader system actions.
The manifest describes a spec-writing skill, but the document also includes installation, bootstrap, repository manipulation, and remote-fetch behavior not reflected in the declared purpose. This mismatch hides materially risky behavior from reviewers and policy engines, which can lead to under-scoped approvals for a skill that performs broader system actions.
The manifest describes a spec-writing skill, but the document also includes installation, bootstrap, repository manipulation, and remote-fetch behavior not reflected in the declared purpose. This mismatch hides materially risky behavior from reviewers and policy engines, which can lead to under-scoped approvals for a skill that performs broader system actions.
Chaining curl output directly into bash combines remote content retrieval and execution into a single unsafe step, eliminating any opportunity for inspection or integrity validation. In a skill that already normalizes shell-heavy behavior, this pattern significantly raises supply-chain compromise risk and could lead to full code execution on the host.
Requires the `adversarial-common` sibling repo (shared engine). One-line install:
curl -fsSL https://raw.githubusercontent.com/chpomob/adversarial-spec/main/scripts/install.sh | bash
or, from an existing checkout:
Allowing an externally supplied --dev-cmd means the pipeline may invoke arbitrary local commands as the writer role. In a skill that already has shell and filesystem interactions, untrusted or loosely validated provider command configuration can become a command-execution vector or route sensitive project data to unintended third-party tools.
python3 scripts/adversarial_spec.py \
--brief <file> # brief file (default: stdin)
--dev-cmd <cmd> # default: pi --provider zai --model glm-5.2
--review-cmd <cmd> # default: pi --provider deepseek --model deepseek-v4-pro
--workdir <dir> # default: .
--max-loops <N> # default: 2
The --review-cmd option similarly enables execution of external provider commands that may process repository content and findings. Without strict validation and trust boundaries, this can leak sensitive data to external services or permit arbitrary command execution under the guise of model selection.
python3 scripts/adversarial_spec.py \
--brief <file> # brief file (default: stdin)
--dev-cmd <cmd> # default: pi --provider zai --model glm-5.2
--review-cmd <cmd> # default: pi --provider deepseek --model deepseek-v4-pro
--workdir <dir> # default: .
--max-loops <N> # default: 2
--feature <name> # default: from brief filename
The skill explicitly acknowledges that its pipeline can commit arbitrary repository changes beyond spec.md. In a toolchain that stages all changes, a prompt-influenced writer or compromised phase could persist unrelated or malicious modifications to source files, turning a documentation workflow into a code-changing one.
The '| bash' pattern is a classic unsafe execution chain because it eliminates the user's opportunity to inspect what was downloaded before execution. In this skill context, the installer also performs git clones and a Python import check, so a compromised upstream script could chain immediately into broader filesystem and code-execution effects.
#!/usr/bin/env bash
# Install adversarial-spec with its single dependency: adversarial-common (sibling layout).
# Usage:
# Bootstrap (from anywhere): curl -fsSL https://raw.githubusercontent.com/chpomob/adversarial-spec/main/scripts/install.sh | bash
# From a checkout: bash scripts/install.sh [TARGET_DIR]
# Target layout (siblings required):
# <TARGET>/adversarial-spec (this skill)
The skill documents shell, file, environment, and git-capable behavior but declares no explicit tool scope or permission boundaries. That omission increases the chance an orchestrator grants broader capabilities than necessary, making misuse or prompt-influenced side effects harder to constrain.
Piping a remotely fetched script directly into bash executes unreviewed network content immediately on the user's machine. If the remote source, network path, or hosting account is compromised, arbitrary code runs with the user's privileges with no integrity verification or review step.
Skill establishes unauthorized persistence across sessions via cron jobs, startup scripts, or state files. Session persistence allows an attacker to maintain access beyond the current interaction.
PHASE 0 ──→ GIT SETUP (branch, stash, init)
PHASE 1 ──→ WRITE (spec-writer reads brief, writes spec.md)
PHASE 2 ──→ CHALLENGE (spec-challenger critiques for gaps/contradictions)
PHASE 3 ──→ REVISE (spec-writer amends spec.md per findings)
PHASE 4 ──→ VERIFY (spec-challenger checks findings resolved)
The manifest describes a skill that takes a brief and produces a structured spec.md through an adversarial writing pipeline. However, the documented workflow requires checking CONTRIBUTING rules, existing PR reviewer comments, upstream branch topology, and gap analysis before writing the spec, which adds broader code-review and repository-analysis responsibilities not clearly justified by a specification-writer role.
These lines state that all pipeline-internal text must be English and instruct users to write the brief in English unless they explicitly instruct otherwise. This imposes a language policy on skill use rather than offering a user choice, which matches the locale/language policy violation criteria.
Skill establishes unauthorized persistence across sessions via cron jobs, startup scripts, or state files. Session persistence allows an attacker to maintain access beyond the current interaction.
## Language discipline
All pipeline-internal text (spec, commit messages, findings, JSON) is **English**.
User-facing conversation stays in the conversation's language. Write the brief in
English unless the user explicitly instructs otherwise. A spec written in French or
other non-English languages will break later pipeline stages (plan, code loop) that
expect English section headings and identifiers.
The text first states precedence as provider-config > explicit role flags > hardcoded fallback commands, which implies explicit flags outrank other non-registry sources. But later it says that without a registry, each role resolves from the explicit flag, then environment variables, then hardcoded fallbacks, creating an inconsistent description of intent and actual resolution order.
The pitfalls section states provider selection is purely configuration-driven, which contradicts earlier documentation that explicit commands, environment variables, and hardcoded fallback commands may be used. This creates a direct mismatch in stated behavior versus documented operational paths.
Skill attempts to fill the context window with filler content, displacing legitimate instructions and safety constraints. This can degrade agent performance or bypass safety boundaries.
| A7 | nit | `_retrospective/ISSUES.md:3` | Stale claim that failures are auto-appended to this file — actually goes to `<out_dir>/ISSUES.md` now |
| A8 | nit | `scripts/adversarial_spec.py:78` | `_ensure_ids` dedup generates awkward compound ids like `S1-3-3` |
| A9 | nit | `scripts/phases/phase_verify.py:55` | Prompt hard-codes `git diff HEAD~1..HEAD` — wrong if writer CLI made multiple commits |
| A10 | nit | `scripts/phases/phase_challenge.py:44` | Full spec embedded in prompt with no size guard — large specs can overflow context window |
## Inspector Findings (Codex GPT-5.6-Sol)
The notes explicitly document that the wrapper defaults to a dangerous permission-skipping mode and only briefly mention using --no-danger for untrusted input. In a security-sensitive, adversarial skill context, documenting an unsafe default without strongly requiring the safer flag increases the chance that operators will run untrusted model prompts with elevated permissions, enabling unintended filesystem or command effects.
The manifest describes a skill that takes a brief and produces a structured spec.md in a git-aware workflow. This file additionally implements an opt-in external research pipeline that gathers authoritative external context and feeds it into the writer, which is a distinct network/external-information capability not stated in the manifest description.
When --deep-research is enabled, the code sends the full brief text to an external research provider. Briefs may contain proprietary plans, credentials, internal URLs, or other sensitive context, and this transfer occurs without any visible consent prompt, redaction step, or in-file user warning at the point of use.
This shell script clones remote GitHub repositories into the user's skills directory, which both transmits network metadata and writes files locally. Although there are status echoes, there is no confirmation prompt or explicit warning in the script comments/output that running it will fetch code from the network and modify the local filesystem.
This code performs a repository commit after modifying spec.md, which is a safety-relevant file/system state change. While the module docstring mentions that the file is edited and committed, there is no user-facing confirmation prompt, log/print disclosure, or inline warning near the operation itself in this code path.
Skill establishes unauthorized persistence across sessions via cron jobs, startup scripts, or state files. Session persistence allows an attacker to maintain access beyond the current interaction.
- file: adversarial_common/report.py
description: "Add quota metadata to the final report (which provider was used, quota state at decision time)."
- file: adversarial_common/jsonio.py
description: "No changes needed — already handles JSON read/write used by check-ai-quota.py output."
---
# Quota-Aware Provider Registry
The spec states that all error messages and report fields must be in English, which imposes a language policy on user-facing output at least for errors. This is a natural-language policy concern because it forces a specific language rather than offering a user choice or clearly limiting the behavior to a justified region-specific context.
No suspicious patterns detected.