agentscope-test-fixture-generator

• Outcome: Review
• Latest audit: Oct 3, 2026 · 2:18 AM UTC
• Version: 1.0.0

Overview

This fixture generator is mostly coherent, but it can silently write a real OpenAI API key into generated test files despite saying it does not embed real credentials.

Review before installing or running. Do not run this skill in an environment where OPENAI_API_KEY is set unless the script is fixed to always emit a placeholder or redact secrets; generated fixtures should be checked before committing, uploading to CI artifacts, or sharing.

A.I.G

Vulnerability Patterns

• Insecure Skill Coding Practices: Finds exploitable flaws such as hardcoded secrets or command injection
• Skill Instruction Hijacking: Alters the agent's session goals or safety constraints when the skill loads
• Agent Memory Poisoning: Writes attacker-controlled rules into memory that affect later sessions
• Remote Payload Retrieval and Execution: Fetches external code whose behavior can change after review
• Embedded Malicious Code: Ships malicious scripts inside the skill and executes them locally

Findings (1)

T09 · Insecure Skill Coding Practices

Error

• Location: scripts/generate_fixtures.py:15

Finding

Real OpenAI API Key Written to Plaintext Test Fixtures

Content

View full analysis

Vulnerability Details

File Location: scripts/generate_fixtures.py, lines 15–22
Vulnerability Type: Plaintext credential exposure
Risk Level: High

Vulnerable Code

python

def agent_config_fixture(name: str, model: str = "gpt-4o") -> dict:
    return {
        "name": name,
        "type": "TextAgent",
        "model_config": {
            "model": model,
            "api_key": os.environ.get("OPENAI_API_KEY", "test-api-key"),
        },
        "sys_prompt": f"You are {name}, a helpful assistant for testing.",
    }

Generated configurations are subsequently serialized to plaintext fixture files, including:

python

write(
    output_dir / "sequential" / "pipeline.json",
    json.dumps(pipeline, indent=2, ensure_ascii=False),
)

Technical Analysis

agent_config_fixture() reads the real OPENAI_API_KEY from the process environment whenever it is available. The resulting value is inserted directly into each generated agent's model_config.

The sequential, parallel, and team fixture generators place these configurations into pipeline dictionaries and serialize them as plaintext JSON under the user-selected output directory. The all pattern invokes all affected generators.

This behavior crosses a trust boundary by moving a secret from protected process environment state into ordinary test-fixture files. It also contradicts the security statement in SKILL.md that generated fixtures do not embed real credentials. Test fixtures are commonly committed to source control, retained as CI artifacts, or shared with other developers, making them an inappropriate destination for live credentials.

No evidence indicates that the key is transmitted to an attacker-controlled endpoint or that the author intended credential theft. The issue is therefore an insecure, reachable implementation flaw rather than confirmed malicious behavior.

Attack Path

1. A developer or CI ...[truncated 1067 chars]

Remediation

View remediation

Remediation Suggestions

• Never copy OPENAI_API_KEY or another live credential into generated fixtures.
• Always emit an inert placeholder such as test-api-key, or emit a non-secret environment-variable reference that the consuming test resolves at runtime.
• If optional live-provider validation is required, read the credential only in memory during the validation operation and never include it in serialized configuration, logs, exceptions, or generated files.
• Add a regression test that sets OPENAI_API_KEY to a sentinel secret, generates every supported pattern, and verifies that the sentinel does not appear anywhere in the output tree.
• Update SKILL.md so its credential-handling documentation accurately reflects the implementation and clearly separates mock fixture generation from any optional live-provider validation workflow.

SkillSpector

Vulnerability Patterns

• Excessive Agency: Unrestricted Tool Access, Autonomous Decision Making, Scope Creep
• MCP Least Privilege: Underdeclared Capability, Wildcard Permission, Missing Permission Declaration
• MCP Tool Poisoning: Hidden Instructions, Unicode Deception, Parameter Description Injection
• Prompt Injection: Instruction Override, Hidden Instructions, Exfiltration Commands
• Data Exfiltration: External Transmission, Env Variable Harvesting, File System Enumeration

Findings (3)

Undeclared Tool Scope

Medium

• Category: MCP Least Privilege
• Confidence: 90% confidence

Finding

The skill declares executable capabilities through its documented usage and metadata requirements (environment variables, running python3, and writing generated fixture files) but does not define any explicit tool scope such as permissions or allowed-tools. This creates an authorization gap where an agent runtime may permit broader file and environment access than intended, increasing the risk of unintended file writes, reading sensitive workspace data, or exposing environment variables during execution.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

Medium

• Category: Not specified by scanner
• Confidence: 96% confidence

Finding

This code is intended to generate test fixtures and example configurations, but it may embed a live API credential instead of mock data. In the skill context, that is more dangerous because fixtures are likely to be committed, shared with developers, or stored in CI artifacts where secrets do not belong.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Medium

• Category: Not specified by scanner
• Confidence: 98% confidence

Finding

The fixture generator pulls OPENAI_API_KEY from the environment and places it into generated JSON fixtures. This can leak a real credential into source trees, test artifacts, CI outputs, or logs, turning mock data generation into secret exfiltration by accident.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.