Back to skill

Security audit

Dark-Factory-Agent

Security checks for vulnerabilities and agentic risk

Overview

This skill does not show exfiltration or persistence, but it can produce misleading success and security reports from mock, random results.

Review carefully before installing or using this in any approval, CI/CD, security, or provenance workflow. Treat generated outcome reports as mock demonstrations only, not as proof that code was generated, tested, secured, or authentically signed.

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (4)

T09 · Insecure Skill Coding Practices

Error
Location
scripts/behavioral_test_engine.py:25
Finding

Randomized mock results are presented as executed behavioral, unit, and integration test evidence

Content
View full analysis
dict: """ Mock execution of a behavioral scenario. In production, this is replaced by the AI agent executing the scenario. """ start = time.time() # Simulate execution time (5–80ms) time.sleep(random.uniform(0.005, 0.08)) duration_ms = round((time.time() - start) * 1000, 2) expected = scenario.get("expected_output", {}) # Simulate a 96% pass rate for demo purposes passed = random.random() > 0.04 return { "passed": passed, "actual_output": expected if passed else {"error": "mock_execution_divergence"}, "duration_ms": duration_ms, "match_score": 1.0 if passed else round(random.uniform(0.3, 0.7), 3) } ``` ```python def _generate_code(spec: dict) -> dict: """ AI agent code generation integration point. In production, this calls the configured LLM/agent to generate code from the specification. Returns a placeholder structure here. """ log("Generating code from specification (AI agent integration point)...") time.sleep(random.uniform(0.1, 0.3)) # Simulate generation time return { "language": "python", "files": [ { "path": f"{spec.get('specification_id', 'output')}_implementation.py", "content": f"# Auto-generated implementation for: {spec.get('title', 'Unknown')}\n# Specification ID: {spec.get('specification_id', 'N/A')}\n# Generated at: {datetime.now(timezone.utc).isoformat()}\n\n# TODO: Replace with AI-generated implementation\npass\n", "language": "python" } ], "generation_metadata": { "model": "agent-integra ...[truncated 5462 chars]
Remediation
View remediation

T09 · Insecure Skill Coding Practices

Error
Location
scripts/orchestrator.py:95
Finding

Unkeyed SHA-256 checksum is misrepresented as a cryptographic signature

Content
View full analysis
dict: """Generate a cryptographic signature for the outcome report.""" content = json.dumps(report_data, sort_keys=True, separators=(",", ":")) digest = hashlib.sha256(content.encode()).hexdigest() return { "algorithm": "SHA-256", "digest": digest, "signed_at": datetime.now(timezone.utc).isoformat(), "note": "In production, sign with a private key for full non-repudiation" } ``` The documented verification procedure is: ```python import json, hashlib report = json.load(open("outcome_report.json")) sig = report.pop("cryptographic_signature") content = json.dumps(report, sort_keys=True, separators=(",", ":")) digest = hashlib.sha256(content.encode()).hexdigest() print("VALID" if digest == sig["digest"] else "TAMPERED") ``` The output schema describes the value as follows: ```json "cryptographic_signature": { "type": "object", "description": "Cryptographic proof of report integrity", "required": ["algorithm", "digest", "signed_at"], "properties": { "algorithm": { "type": "string", "description": "Hashing/signing algorithm used" }, "digest": { "type": "string", "description": "Hex digest of the report content" }, "signed_at": { "type": "string", "format": "date-time" }, "note": { "type": "string" } } } ``` ### Technical Analysis SHA-256 is a cryptographic hash function, but an unkeyed hash is not a digital signature and does not authenticate the report producer. Every party capable of modifying the report is also capable of recalculating its digest. The documented verifier trusts a digest stored inside the same untrusted document, so it detec ...[truncated 1480 chars]
Remediation
View remediation

T09 · Insecure Skill Coding Practices

Error
Location
scripts/orchestrator.py:179
Finding

Declared security requirements and guards are accepted but never enforced

Content
View full analysis
Remediation
View remediation

T09 · Insecure Skill Coding Practices

Warning
Location
scripts/specification_validator.py:48
Finding

Insufficient type validation permits malformed specifications to crash the validator and orchestrator

Content
View full analysis
Remediation
View remediation
Vulnerability Patterns
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
Findings (5)

Intent-Code Divergence

High
Category
Not specified by scanner
Confidence
99% confidence
Finding

The orchestrator claims to run unit and integration tests, but both functions fabricate randomized pass/fail outcomes instead of executing real tests against generated code or dependencies. This can falsely certify unsafe or nonfunctional outputs as validated, undermining the trustworthiness of the entire pipeline and any downstream deployment or approval decisions based on the report.

Content

No source excerpt is available for this finding.

Autonomous Decision Making

Medium
Category
Excessive Agency
Confidence
90% confidence
Finding

The skill is designed for autonomous decision-making and execution: it validates inputs, generates code, runs tests, and produces outputs without a required human approval checkpoint. In this context, autonomy is dangerous because generated code and test actions can trigger execution of unsafe logic, especially if specifications are attacker-controlled or insufficiently constrained.

Content

Scanner excerpt · SKILL.md (reported line 3)May include surrounding context.

md
---
name: dark-factory
description: Autonomously executes validated specifications from intent-engineering to produce provable, cryptographically signed outcome reports. Use when you need to: validate a specification for clarity and completeness, run behavioral tests against a mock environment, generate code autonomously, execute unit and integration tests, or produce a signed outcome report ready for feedback-loop analysis. Can be used standalone or as Stage 2 in the unified-orchestrator pipeline.
---

# Dark Factory

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

The skill explicitly advertises autonomous code generation and execution of unit/integration tests, but the documentation does not prominently warn users that running this workflow can modify the environment, execute untrusted generated code, or consume sensitive system resources. This creates a real safety and security gap because operators may invoke it without understanding the execution risk surface.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Medium
Category
Not specified by scanner
Confidence
98% confidence
Finding

This script claims to execute behavioral scenarios and produce a test report, but the core logic in _mock_execute fabricates results using randomness rather than validating real agent behavior. In a testing or assurance pipeline, this can misrepresent system quality, generate false confidence, and allow unsafe or noncompliant behavior to be promoted based on meaningless pass metrics.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Medium
Category
Not specified by scanner
Confidence
98% confidence
Finding

The function presents the report as cryptographically signed, but it only computes an unauthenticated SHA-256 digest over the JSON content. Anyone who can modify the report can recompute the digest, so this provides integrity checking at best and no authenticity or non-repudiation despite the surrounding language claiming a signed report.

Content

No source excerpt is available for this finding.