agentscope-model-compare

• Outcome: Review
• Latest audit: Oct 3, 2026 · 2:19 AM UTC
• Version: 1.0.0

Overview

The skill mostly does what it says, but its model-selection code can route a DashScope API key through an OpenAI chat client, creating a real credential-boundary risk.

Review carefully before installing. Use only low-sensitivity prompts, avoid passing secrets or proprietary content as the task, and do not configure a DashScope key until the provider routing is fixed or unknown model names are rejected. Rotate any DashScope key that may already have been used through this fallback path.

A.I.G

Vulnerability Patterns

• Insecure Skill Coding Practices: Finds exploitable flaws such as hardcoded secrets or command injection
• Skill Instruction Hijacking: Alters the agent's session goals or safety constraints when the skill loads
• Agent Memory Poisoning: Writes attacker-controlled rules into memory that affect later sessions
• Remote Payload Retrieval and Execution: Fetches external code whose behavior can change after review
• Embedded Malicious Code: Ships malicious scripts inside the skill and executes them locally

Findings (1)

T09 · Insecure Skill Coding Practices

Error

• Location: scripts/compare_models.py:24

Finding

DashScope Credential Passed to an OpenAI Model Client

Content

View full analysis

Vulnerability Details

File Location: scripts/compare_models.py, lines 24–27
Vulnerability Type: Cross-provider credential disclosure caused by an incorrect model client configuration
Risk Level: High

Vulnerable Code

python

return OpenAIChatModel(
    api_key=os.environ.get("DASHSCOPE_API_KEY", os.environ["OPENAI_API_KEY"]),
    model_name=model_name,
)

Technical Analysis

build_model() routes every model whose name does not start with gpt-, o1-, or o3- into this branch. The branch retrieves DASHSCOPE_API_KEY when available but still constructs an OpenAIChatModel.

No DashScope-specific model adapter or verified DashScope-compatible API endpoint is configured. Consequently, the credential selected for Alibaba Cloud is placed into a client configured for a different provider. When run_task() invokes the resulting agent, the model client can use that credential in its outbound authentication request.

This contradicts the documented purpose of DASHSCOPE_API_KEY in SKILL.md, where it is described as the credential for Alibaba Cloud models. The issue is a reachable provider-boundary error rather than evidence of intentional credential theft.

Attack Path

1. The operator configures DASHSCOPE_API_KEY to compare an Alibaba Cloud model, as supported by the skill documentation.
2. The skill is invoked with a model name that does not start with gpt-, o1-, or o3-.
3. build_model() enters the fallback branch and selects DASHSCOPE_API_KEY.
4. The key is passed to OpenAIChatModel without a DashScope-specific adapter or endpoint.
5. run_task() calls the agent, causing the incorrectly configured model client to make an authenticated model request.
6. The DashScope credential may therefore be disclosed to the endpoint used by the OpenAI client rather than remaining within its intended provider boundary.

Impact Assessment

A reusable DashScope API credential ...[truncated 548 chars]

Remediation

View remediation

Remediation Suggestions

• Use the AgentScope adapter intended for DashScope or Alibaba Cloud models.
• If an OpenAI-compatible client is required, explicitly configure and validate the official DashScope-compatible API endpoint rather than relying on the OpenAI client's default endpoint.
• Represent each provider with a distinct configuration containing its model-name rules, credential variable, client type, and allowed endpoint.
• Reject unknown model names instead of routing every unrecognized model through a credential-bearing fallback.
• Validate before sending a request that the selected credential belongs to the configured endpoint.
• Add tests that mock outbound client configuration and assert that DASHSCOPE_API_KEY can only be associated with an approved DashScope endpoint.
• After correcting the configuration, rotate any DashScope key that may already have been used through the affected path.

SkillSpector

Vulnerability Patterns

• Data Exfiltration: External Transmission, Env Variable Harvesting, File System Enumeration
• MCP Least Privilege: Underdeclared Capability, Wildcard Permission, Missing Permission Declaration
• Prompt Injection: Instruction Override, Hidden Instructions, Exfiltration Commands
• Privilege Escalation: Excessive Permissions, Sudo/Root Execution, Credential Access
• Supply Chain: Unpinned Dependencies, External Script Fetching, Obfuscated Code

Findings (2)

Undeclared Tool Scope

Medium

• Category: MCP Least Privilege
• Confidence: 90% confidence

Finding

The skill declares required environment variables containing API secrets but does not define any explicit tool scope such as permissions or allowed-tools. This weakens least-privilege controls: any code in the skill can potentially access available secrets without a clearly documented or enforceable boundary, increasing the risk of accidental exposure or misuse if the implementation is modified or compromised.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium

• Category: Not specified by scanner
• Confidence: 95% confidence

Finding

The script sends the user-supplied task verbatim to third-party model providers via await agent(task) without any disclosure, consent prompt, or warning that the prompt contents will leave the local environment. This creates a real data exposure risk if users include secrets, proprietary code, or sensitive business information while benchmarking models, especially because the skill is specifically designed to compare multiple remote models and may transmit the same sensitive prompt to several providers.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.