AI App Testing

• Outcome: Review
• Latest audit: Jul 29, 2026 · 3:44 AM UTC
• Version: 1.1.1

Overview

The skill is mostly a disclosed AI testing toolkit, but some security-audit outputs are simulated while presented as compliance results, and online tests can send realistic sensitive test data or retrieved documents to configured endpoints without clear warning.

Review this before installing if you plan to use it for security or compliance decisions. Keep endpoints local or trusted, replace the realistic ID/phone fixtures with clearly fake values, avoid sending private RAG corpus text to third-party judge models, and do not rely on the MCP compliance score as verified evidence without improving those checks.

SkillSpector

Vulnerability Patterns

• Prompt Injection: Instruction Override, Hidden Instructions, Exfiltration Commands
• Data Exfiltration: External Transmission, Env Variable Harvesting, File System Enumeration
• Taint Tracking: Direct Taint Flow, Variable-Mediated Taint Flow, Credential Exfiltration Chain
• MCP Least Privilege: Underdeclared Capability, Wildcard Permission, Missing Permission Declaration
• MCP Tool Poisoning: Hidden Instructions, Unicode Deception, Parameter Description Injection

Findings (9)

Tainted flow: 'doc_text' from requests.post (line 72, network input) → requests.post (network output)

Medium

• Category: Data Flow

Content

r=requests.post(self.rag_ep, json={"query":item["q"]}, timeout=30)
                data=r.json(); response=data.get("response","")
                docs=data.get("retrieved_docs",[]); doc_text="\n".join([d.get("content","") for d in docs])
                jr=requests.post(self.chat_ep, json={"model":self.model,
                    "messages":[{"role":"user","content":judge_prompt.format(docs=doc_text,query=item["q"],response=response)}]}, timeout=15)
                jr_data=jr.json()["choices"][0]["message"]["content"]
                m=re.search(r"\{.*\}",jr_data,re.DOTALL)

• Confidence: 91% confidence
• Finding: Untrusted content returned from the RAG endpoint (`retrieved_docs` and model `response`) is embedded directly into a second LLM judge prompt and sent to `chat_ep`. This creates a prompt-injection and data-exfiltration path: malicious retrieved content can steer the judge model, corrupt evaluation results, or cause sensitive retrieved text to be retransmitted to another external service.

Lp3

Medium

• Category: MCP Least Privilege
• Confidence: 91% confidence
• Finding: The skill advertises and instructs use of runnable Python scripts that can read/write files and make network requests, but the metadata shown does not declare corresponding permissions or capability boundaries. This creates a trust and review gap: users or platforms may execute a skill with broader effective access than its manifest communicates, increasing the chance of unintended data exposure, unsafe filesystem changes, or outbound requests to arbitrary endpoints.

Description-Behavior Mismatch

High

• Confidence: 98% confidence
• Finding: The tool presents itself as an MCP audit/tester, but core audit outputs are hard-coded or simulated: capability support is always left untested, API compliance is unconditionally marked true, and scoring treats fabricated data as real evidence. This can mislead users into trusting a server's security/compliance posture and create false assurance in security-sensitive testing workflows.

Intent-Code Divergence

Medium

• Confidence: 95% confidence
• Finding: The generated report is labeled as a compliance audit report even though major sections are not actually verified and some values are inferred or hard-coded. In a security testing skill, this is dangerous because downstream users may rely on the report for operational or governance decisions despite its unsupported conclusions.

Missing User Warnings

Medium

• Confidence: 96% confidence
• Finding: The test scenario hard-codes highly sensitive personal data, including a phone number and a national ID number, and the tool later sends all prompts to a configured HTTP endpoint. Even if intended as test data, this creates a realistic risk of PII leakage to local logs, remote model providers, proxies, or telemetry systems, and normalizes unsafe handling of regulated data in test suites.

Missing User Warnings

Medium

• Confidence: 95% confidence
• Finding: The session tester transmits full conversation contents to an arbitrary endpoint without any runtime disclosure, consent prompt, or safeguard. Because the conversations can include sensitive business or personal content, users may unknowingly exfiltrate data to external services, especially if the endpoint is changed from localhost to a remote host.

Ssd 1

Medium

• Confidence: 96% confidence
• Finding: The evaluator prompt directly interpolates untrusted context, question, and model response into a single judge instruction without delimiting or defensive framing. An attacker can place prompt-injection text inside those fields to manipulate the judge model's scoring or output format, undermining evaluation integrity and potentially causing false pass/fail decisions in automated testing pipelines.

External Transmission

Medium

• Category: Data Exfiltration

Content

t0 = time.time()
            conversation.append({"role":"user","content":text})
            try:
                r = requests.post(self.endpoint, json={
                    "model":self.model,"messages":conversation,
                    "temperature":0,"max_tokens":512
                }, timeout=self.timeout)

• Confidence: 93% confidence
• Finding: This code performs external transmission of conversation data via HTTP POST, which is security-relevant because the payload contains cumulative multi-turn content and may include sensitive information. In a testing skill, outbound network access is expected, but it still becomes a real vulnerability when combined with arbitrary endpoint configuration and lack of user warning, endpoint validation, or data minimization.

External Transmission

Medium

• Category: Data Exfiltration

Content

r=requests.post(self.rag_ep, json={"query":item["q"]}, timeout=30)
                data=r.json(); response=data.get("response","")
                docs=data.get("retrieved_docs",[]); doc_text="\n".join([d.get("content","") for d in docs])
                jr=requests.post(self.chat_ep, json={"model":self.model,
                    "messages":[{"role":"user","content":judge_prompt.format(docs=doc_text,query=item["q"],response=response)}]}, timeout=15)
                jr_data=jr.json()["choices"][0]["message"]["content"]
                m=re.search(r"\{.*\}",jr_data,re.DOTALL)

• Confidence: 88% confidence
• Finding: This call sends retrieved document content and generated answers to a separate chat model endpoint. In context, that means potentially sensitive RAG corpus material is retransmitted outside the primary service boundary, and because the content is untrusted, it can also manipulate the judge model and invalidate security/quality assessments.

Static analysis

No suspicious patterns detected.