Back to skill

Security audit

Artifact Contract Auditor

Security checks for vulnerabilities and agentic risk

Overview

The skill mostly performs the advertised audit, but it has under-disclosed extra writes and path handling that can reach outside the expected workspace or pipeline directory.

Install only if you are comfortable running it on trusted workspaces. Before use, treat `PIPELINE.lock.md` and `output/` as trusted inputs, avoid symlinked workspace directories, and expect it may update both `output/CONTRACT_REPORT.md` and `output/QUALITY_GATE.md` despite the narrower description.

Vulnerability Patterns
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Findings (2)

T05 · Unauthorized Access and Privilege Escalation

Warning
Location
scripts/run.py:69
Finding
Workspace-Controlled Pipeline Lock Allows Reads Outside the Bundled Pipelines Directory<![CDATA[ ## Vulnerability Details **File Location**: `scripts/run.py:23-30, 69-78`; `tooling/pipeline_spec.py:62-64, 145-157` **Vulnerability Type**: Path traversal and unrestricted absolute-path resolution **Risk Level**: Medium ### Vulnerable Code ```python def _read_pipeline_path(workspace: Path) -> str: lock_path = workspace / "PIPELINE.lock.md" if not lock_path.exists(): return "" for raw in lock_path.read_text(encoding="utf-8", errors="ignore").splitlines(): line = raw.strip() if line.startswith("pipeline:"): return line.split(":", 1)[1].strip() return "" ``` ```python pipeline_rel = _read_pipeline_path(workspace) pipeline_path = (repo_root / pipeline_rel).resolve() if pipeline_rel else None target_artifacts: list[tuple[str, bool]] = [] pipeline_load_error = "" if pipeline_path and pipeline_path.exists(): try: from tooling.pipeline_spec import PipelineSpec spec = PipelineSpec.load(pipeline_path) ``` The resulting path is opened by the pipeline parser: ```python @staticmethod def load(path: Path) -> "PipelineSpec": resolved = path.resolve() raw_frontmatter, frontmatter = _load_variant_aware_frontmatter(resolved) ``` ```python def _load_variant_aware_frontmatter( path: Path, seen: set[Path] | None = None, ) -> tuple[dict[str, Any], dict[str, Any]]: resolved = path.resolve() active = set(seen or set()) if resolved in active: chain = " -> ".join(str(p) for p in [*active, resolved]) raise ValueError(f"Cyclic `variant_of` chain detected: {chain}") active.add(resolved) text = resolved.read_text(encoding="utf-8") raw = _parse_frontmatter(text) ``` ### Technical Analysis The `pipeline:` value is read from `PIPELINE.lock.md` inside a caller-supplied workspace. The value is joined to `repo_root` and resolved, but the resolved path is never checked to ensure that it remains beneath the expected `pipelines/` directory. `pathlib` ...[truncated 2206 chars]
Remediation
<![CDATA[ ## Remediation Suggestions 1. Reject absolute values from `PIPELINE.lock.md`. 2. Resolve pipeline references exclusively against the expected directory. 3. Verify containment after canonicalization: ```python pipelines_root = (repo_root / "pipelines").resolve() supplied = Path(pipeline_rel) if supplied.is_absolute(): raise ValueError("Absolute pipeline paths are not allowed") candidate = (repo_root / supplied).resolve() if not candidate.is_relative_to(pipelines_root): raise ValueError("Pipeline path must remain under the pipelines directory") if not candidate.name.endswith(".pipeline.md"): raise ValueError("Pipeline specification must end with .pipeline.md") ``` 4. Prefer resolving a pipeline identifier through the existing centralized resolver rather than directly joining the untrusted value to `repo_root`. 5. Allowlist the bundled pipeline specifications if arbitrary pipeline files are not required. 6. Avoid placing raw parser exceptions into user-facing reports. Use a generic failure message and place sanitized diagnostic details in a protected local log. 7. Apply equivalent containment validation to `variant_of` references in `tooling/pipeline_spec.py`, including rejection of absolute references. ]]>

T09 · Insecure Skill Coding Practices

Warning
Location
scripts/run.py:198
Finding
Symlinked Output Directory Allows Report Writes Outside the Workspace<![CDATA[ ## Vulnerability Details **File Location**: `scripts/run.py:198-200`; `tooling/quality_gate.py:1395-1400, 1437-1441`; `tooling/common.py:25-38` **Vulnerability Type**: Symlink-based path escape and unrestricted file replacement **Risk Level**: Medium ### Vulnerable Code The contract report is written to a path derived from the workspace: ```python out_path = workspace / "output" / "CONTRACT_REPORT.md" ensure_dir(out_path.parent) atomic_write_text(out_path, "\n".join(lines).rstrip() + "\n") ``` The quality report uses the same unvalidated output directory: ```python def write_quality_report( *, workspace: Path, unit_id: str, skill: str, issues: list[QualityIssue], ) -> Path: from tooling.common import atomic_write_text, ensure_dir ensure_dir(workspace / "output") report_path = workspace / "output" / "QUALITY_GATE.md" ``` Existing quality-report contents are read and rewritten: ```python if report_path.exists() and report_path.stat().st_size > 0: prev = report_path.read_text( encoding="utf-8", errors="ignore", ).rstrip() + "\n\n---\n\n" atomic_write_text(report_path, prev + entry) else: atomic_write_text(report_path, entry) ``` The atomic-write helper does not reject symlinked parent directories or verify workspace containment: ```python def ensure_dir(path: Path) -> None: path.mkdir(parents=True, exist_ok=True) def atomic_write_text(path: Path, content: str) -> None: ensure_dir(path.parent) fd, tmp_path = tempfile.mkstemp(prefix=path.name, dir=str(path.parent)) try: with os.fdopen(fd, "w", encoding="utf-8") as handle: handle.write(content) os.replace(tmp_path, path) except Exception: try: os.unlink(tmp_path) except OSError: pass raise ``` ### Technical Analysis The workspace is supplied through the `--workspace` command-line option, and the implementation assumes that its `output` ...[truncated 2105 chars]
Remediation
<![CDATA[ ## Remediation Suggestions 1. Resolve and validate the workspace before any access: ```python workspace = Path(args.workspace).resolve(strict=True) if not workspace.is_dir(): raise ValueError("Workspace must be a directory") ``` 2. Reject a symlinked `output` entry before creating or writing reports: ```python output_dir = workspace / "output" if output_dir.is_symlink(): raise ValueError("Workspace output directory must not be a symlink") output_dir.mkdir(parents=True, exist_ok=True) resolved_output = output_dir.resolve(strict=True) if not resolved_output.is_relative_to(workspace): raise ValueError("Output directory escapes the workspace") ``` 3. Validate the final report parent immediately before both temporary-file creation and replacement to reduce time-of-check/time-of-use exposure. 4. Where supported, use descriptor-relative filesystem operations and no-follow flags such as `O_NOFOLLOW`. 5. Open a trusted output-directory file descriptor and perform report operations relative to that descriptor rather than repeatedly resolving path strings. 6. Reject symlinked path components throughout the workspace path when processing untrusted workspaces. 7. Apply the same safe-write policy centrally in `atomic_write_text()` or introduce a workspace-scoped safe-write helper so every report writer receives consistent protection. 8. Add regression tests covering: - A symlinked `output` directory. - Replacement of `output` between validation and writing. - A workspace path containing symlinked ancestors. - Existing external files named `CONTRACT_REPORT.md` and `QUALITY_GATE.md`. ]]>
Vulnerability Patterns
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
Findings (43)

Tp4

High
Category
MCP Tool Poisoning
Confidence
96% confidence
Finding
The primary behavior matches the description closely: it reads workspace state, checks DONE-unit outputs and required pipeline target artifacts, produces PASS/FAIL/OK status, and writes output/CONTRACT_REPORT.md without network access. However, the description explicitly says 'analysis-only; do not edit content artifacts; only write the report.' The code goes beyond that by importing write_quality_report and conditionally writing a quality report entry (noted in comments as keeping QUALITY_GATE.md updated). That is a material undeclared write side effect and conflicts with the stated guardrail and the specific claim that it writes only output/CONTRACT_REPORT.md.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding
The declared skill is a narrow, analysis-only contract auditor that should inspect workspace completeness and write just output/CONTRACT_REPORT.md. The supplied code does not implement a contract audit or report generation. Instead, it is a general-purpose workspace helper module with many mutating capabilities: atomic file writes, backups, YAML/TSV/JSONL serialization, tree copying, status updates, decisions approval checklist management, checkpoint block insertion, and query seeding based on topics and pipeline settings. These behaviors materially exceed and differ from the declared purpose and violate the stated guardrail of only writing the report.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding
The declared description is for an analysis-only contract auditor that checks DONE outputs versus target_artifacts and writes output/CONTRACT_REPORT.md. The supplied code does something materially different: it is an execution engine for pipeline units. It reads and mutates workspace control files, changes statuses to DOING/DONE/BLOCKED/TODO, may auto-approve checkpoints in DECISIONS.md, launches scripts via subprocess, writes logs/error reports, and enforces runtime quality/cutover rules. There is no implementation of a contract audit, no comparison against target_artifacts, and no writing of output/CONTRACT_REPORT.md. This is a clear description-behavior mismatch.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding
The declared description is for a contract-audit skill whose purpose is to verify workspace completeness against pipeline artifact requirements and emit a single CONTRACT_REPORT.md, with guardrails limiting behavior to analysis-only reporting. The supplied code does something materially different: it is an ideation/report-generation module for creating research directions from literature notes and workspace briefs. It includes functions for parsing an idea brief, resolving an ideation contract from pipeline specs, clustering notes, generating IdeaSignal/DirectionCard/ScreenedDirection objects, scoring directions with rubrics, and producing brainstorming memo/appendix/report markdown and structured payloads. While both interact with workspace files and pipeline specs, the primary purpose, inputs, outputs, and capabilities are unrelated to artifact-contract auditing. This is a clear description-behavior mismatch.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding
The declared description is for an analysis tool that inspects a workspace's artifact completeness and emits a PASS/FAIL report at output/CONTRACT_REPORT.md. The supplied code chunk does something materially different: it defines data models and helper functions to load pipeline specification files, parse YAML front matter, validate fields, resolve variant references, and merge overrides. While target_artifacts appears as one parsed field in PipelineSpec, the code never audits the workspace, never checks DONE outputs, never compares actual artifacts to contract expectations, and never writes the promised report. This is a clear description-behavior mismatch with a different primary purpose.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding
There is a clear description-behavior mismatch. The declared purpose is a specific end-of-run workspace audit that evaluates whether required artifacts exist and emits a PASS/FAIL report to output/CONTRACT_REPORT.md. The actual code chunk contains only helper functions for text processing and structured content parsing: slugifying unit IDs, loading outline sections from YAML, extracting subsection units, mapping JSONL records, reading bibliography keys, deduplicating strings, cleaning excerpts, formatting citation phrases, splitting Markdown into heading blocks, and dumping JSONL lines. These are materially different capabilities and do not show the declared audit logic, report generation, or artifact completeness checks. While the code has no obvious forbidden side effects or network behavior, its primary purpose is unrelated to the stated skill description.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding
The declared description is narrowly scoped: audit workspace completeness against the pipeline artifact contract (`DONE` outputs + target_artifacts), write only `output/CONTRACT_REPORT.md`, no edits beyond that, and provide an auditable PASS/FAIL completeness snapshot. The supplied code does something materially different and much broader. It defines a generic `quality_gate.py` with dispatch for many skills (`idea-brief`, `literature-engineer`, `section-bindings`, `writer-selfloop`, `latex-compile-qa`, `artifact-contract-auditor`, etc.), performs deep semantic/content checks on many artifact types, enforces policy/profile-specific thresholds, and checks quality properties like placeholders, repeated boilerplate, evidence density, citation verification, and outline structure. It also contains a report writer that writes `output/QUALITY_GATE.md`, not the declared `output/CONTRACT_REPORT.md`. Although some of this could support auditing, the primary behavior shown is a multi-skill quality gate system rather than a narrow artifact-contract auditor. That is a substantive description/behavior mismatch.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding
The declared description frames the skill as a narrow contract audit: verify workspace completeness against pipeline artifact contract metadata and write a single `output/CONTRACT_REPORT.md`, with an analysis-only PASS/FAIL purpose. The supplied code instead implements a substantial multi-artifact quality gate with highly specific validators for subsection briefs, chapter briefs, coverage reports, evidence drafts, anchor sheets, writer context packs, evidence bindings, visuals, transitions, writer self-loop reports, eval anchor reports, section manifests, merged drafts, and more. These checks go far beyond completeness/self-containedness: they inspect semantic and stylistic properties of prose, citation density and validity, structural depth, placeholder leakage, report PASS markers, and consistency with bibliography/evidence mappings. While this is still analysis-only and uses no network, its primary purpose is materially broader and different from a simple pipeline artifact contract audit, and the declared report/output focus does not match the behavior shown.

Tp4

High
Category
MCP Tool Poisoning
Confidence
99% confidence
Finding
This is a clear description-behavior mismatch. The declared skill is narrowly framed as an artifact contract auditor whose job is to assess DONE outputs and target_artifacts, writing `output/CONTRACT_REPORT.md` and otherwise remaining analysis-only. While the snippet does contain `_check_contract_report`, most of the provided code implements a general quality gate for research-writing artifacts. It analyzes draft prose structure, citation placement, boilerplate repetition, global review sections, protocol/tutorial specs, LaTeX scaffold validity, compilation success, undefined citations, PDF page count, and placeholder text. Those are materially broader and different capabilities than simply verifying workspace completeness against a pipeline artifact contract. Resource access also exceeds the declared scope because it reads many files across output/, citations, latex/, logs, and PDFs. The description would only fit `_check_contract_report`, not the supplied code chunk as a whole.

Context-Inappropriate Capability

High
Confidence
96% confidence
Finding
This shared utility module exposes numerous write and mutation primitives such as atomic file writes, backups, YAML/TSV/JSONL saves, tree copies, and decision/status updates that can modify many workspace artifacts. For a skill whose declared guardrail is analysis-only and only writing output/CONTRACT_REPORT.md, bundling broad mutation helpers creates capability drift: if imported or reused by the skill flow, the skill can alter unrelated files despite its contract.

Scope Creep

High
Confidence
98% confidence
Finding
The code supports direct modification of files well beyond the sole permitted artifact, including status logs, decisions files, query seeds, YAML dumps, tabular saves, backups, and recursive copy operations. In the context of an audit skill, this is especially dangerous because users and orchestrators may trust it as non-destructive; a compromised or buggy execution path could silently rewrite workflow state, approvals, or inputs under the guise of producing a report.

Description-Behavior Mismatch

High
Confidence
99% confidence
Finding
The file implements a pipeline runner and state manager: it selects runnable units, updates statuses, handles approvals, executes scripts, performs quality gating, and invalidates downstream work. That behavior materially differs from an artifact contract auditor, so the skill presents a deceptive capability surface and can perform far more powerful actions than the description implies.

Scope Creep

High
Confidence
97% confidence
Finding
The code changes UNITS.csv, STATUS.md, and DECISIONS.md state, including marking units DOING/DONE/BLOCKED and auto-approving checkpoints. That exceeds the manifest's guardrail of analysis-only and 'only write the report,' making the skill capable of mutating workflow control state and potentially advancing a pipeline without proper review.

Context-Inappropriate Capability

High
Confidence
98% confidence
Finding
This logic executes pipeline units by invoking scripts/run.py for whatever unit is marked runnable, effectively operating as a task executor rather than an artifact-contract auditor. In the stated skill context, that is a dangerous scope mismatch because invoking skills can run untrusted or unexpected code while the user expects passive auditing only.

Description-Behavior Mismatch

High
Confidence
99% confidence
Finding
This file implements a full research-ideation and memo-generation pipeline that is unrelated to the declared skill purpose of artifact contract auditing. In a skill advertised as analysis-only and limited to writing `output/CONTRACT_REPORT.md`, hidden unrelated logic materially expands capability and trust assumptions, creating a supply-chain style risk that the skill can be invoked or repurposed to process workspace data and generate undeclared outputs.

Scope Creep

High
Confidence
98% confidence
Finding
The code contains generic JSON/JSONL/Markdown writers that can create or overwrite multiple artifacts, which exceeds the skill contract stating it should only write the contract report. In the context of an agent skill, undeclared write primitives increase the chance of accidental or intentional workspace modification, data sprawl, and artifact forgery that bypasses user expectations and artifact-audit controls.

Lp3

Medium
Category
MCP Least Privilege
Confidence
92% confidence
Finding
The skill declares required binaries and clearly intends to read files, write a report, and invoke a Python script, but it does not declare an explicit tool/permission scope. In an agent platform, missing scope boundaries can cause the runner to grant broader file or shell capabilities than users expect, increasing the blast radius if the implementation is later changed or is inconsistent with the documentation.

Vague Triggers

Medium
Confidence
93% confidence
Finding
This markdown/manifest-style file declares generic activation hints such as "survey", "review", and "literature review" while also setting `routing_default: true`. Those phrases overlap with common research-related requests and the file does not provide exclusion conditions or negative examples to clarify when this specific arXiv survey pipeline should or should not activate.

Natural-Language Policy Violations

Medium
Confidence
95% confidence
Finding
The title and core audience/scope repeatedly define the pipeline as specifically for '中文毕业论文' and frame the workflow around Chinese-language thesis production. This is a natural-language locale constraint, and the file does not offer the user a language choice or explicit opt-in mechanism within the skill description.

Vague Triggers

Medium
Confidence
93% confidence
Finding
The pipeline declares very broad routing hints such as 'idea', 'ideation', 'brainstorm', and generic Chinese equivalents. These terms can match ordinary user requests and cause the agent to invoke this pipeline unexpectedly, creating unintended workspace writes and pipeline execution in contexts where the user did not ask for a contract-driven research workflow.

Vague Triggers

Medium
Confidence
92% confidence
Finding
The routing_hints include broad English phrases like "snapshot", "one-page", and "one page", which are common in ordinary user requests and can cause this pipeline to activate when the user did not explicitly intend a literature-snapshot workflow. Misrouting can lead to unintended execution of multiple downstream skills and generation of workspace artifacts, creating workflow confusion and potentially causing analysis to run on the wrong task context.

Vague Triggers

Medium
Confidence
92% confidence
Finding
The routing hint includes the generic phrase "peer review," which can match broad user requests and cause this pipeline to activate outside its intended narrow context. That can misroute tasks to a workflow that expects manuscript-audit artifacts and produces review outputs, increasing the risk of incorrect execution, confusion, or unintended file generation in unrelated review scenarios.

Vague Triggers

Medium
Confidence
95% confidence
Finding
The routing_hints list includes very generic terms such as "systematic review" and especially "systematic", which can cause the skill or pipeline to be selected for loosely related prompts that merely contain those words. Overbroad routing increases the chance of unintended activation, misrouting user tasks, and accidental execution of analysis workflows in contexts where they were not requested.

Vague Triggers

Medium
Confidence
88% confidence
Finding
The routing hint list includes broad terms like "tutorial" and the generic non-English equivalent "教程", which can cause this pipeline to activate on vague requests that merely mention tutorials rather than explicitly asking for this artifact workflow. Misrouting can lead the agent to run an inappropriate pipeline, produce unintended workspace artifacts, and give users a misleading sense that the request was correctly interpreted.

Scope Creep

Medium
Confidence
97% confidence
Finding
The skill metadata and guardrail state it should only write `output/CONTRACT_REPORT.md`, but this code also conditionally writes a quality report via `write_quality_report`, likely affecting `QUALITY_GATE.md`. That is a contract/behavior mismatch: downstream users or orchestrators may trust the declared write scope and be surprised by additional file modifications, which can undermine auditability and least-privilege assumptions.

Static analysis

No suspicious patterns detected.