Back to skill

Security audit

unisound-diagnosis-sufficiency-review

Security checks for vulnerabilities and agentic risk

Overview

This medical-record review skill is mostly purpose-aligned, but it can send sensitive records and bearer credentials to configurable network endpoints and can write record-derived data to disk.

Install only in an environment where medical records are already de-identified, outbound network access is restricted to approved guideline and model services, and users understand that LLM mode may send record excerpts externally. Prefer --no-llm for offline review, avoid arbitrary --base values, and avoid --save-prepared or output-file options unless the destination is access-controlled and retention is managed.

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (3)

T09 · Insecure Skill Coding Practices

Error
Location
scripts/diagnosis_sufficiency_review.py:481
Finding
Attacker-Controlled LLM Endpoint Can Receive Medical Records and Bearer Credentials<![CDATA[ ## Vulnerability Details **File Location**: `scripts/diagnosis_sufficiency_review.py:481-510, 720-727`; `scripts/run.py:160, 184` **Vulnerability Type**: Unrestricted outbound destination, credential disclosure, and server-side request forgery **Risk Level**: High ### Vulnerable Code ```python class HardcodedLlmClient: def __init__(self, llm_settings: LlmSettings) -> None: self._settings = llm_settings def complete(self, prompt: str, model_name: str | None = None) -> str: selected_model = model_name or self._settings.default_model model_config = self._settings.models.get(selected_model) if model_config is None: raise ValueError(f"未找到模型配置: {selected_model}") payload = { "model": model_config.model_id or selected_model, "messages": [{"role": "user", "content": prompt}], "temperature": model_config.temperature, } if model_config.type == "openai_compatible": return self._post_chat( url=f"{model_config.base_url.rstrip('/')}/chat/completions", payload=payload, headers={"Authorization": f"Bearer {model_config.api_key}"}, ) raise ValueError(f"不支持的模型类型: {model_config.type}") def _post_chat(self, url: str, payload: dict[str, Any], headers: dict[str, str]) -> str: body = json.dumps(payload, ensure_ascii=False).encode("utf-8") req = request.Request( url=url, data=body, headers={"Content-Type": "application/json", **{key: value for key, value in headers.items() if value}}, method="POST", ) opener = request.urlopen(req) if not self._settings.timeout else request.urlopen(req, timeout=self._settings.timeout) ``` The endpoint is populated directly from the request payload: ```python llm_client = _load_llm_client( use_llm, str(payload.get("appkey") or "").strip(), str(payload.get(" ...[truncated 2798 chars]
Remediation
<![CDATA[ ## Remediation Suggestions 1. Remove the caller-controlled `base` option in production and use a deployment-controlled endpoint. 2. If endpoint configurability is required, enforce an exact allowlist of approved HTTPS origins. 3. Parse and validate the URL before use: - Require the `https` scheme. - Reject embedded credentials and unexpected ports. - Reject loopback, private, link-local, multicast, and reserved addresses after DNS resolution. - Revalidate resolved addresses when connecting to mitigate DNS rebinding. 4. Disable redirects or validate every redirect destination against the same allowlist. 5. Bind credentials to a specific configured origin and never attach an LLM credential to an arbitrary URL. 6. Use narrowly scoped, short-lived credentials and rotate any credential that may have been sent to an untrusted destination. 7. Add tests proving that HTTP, localhost, private-network, metadata-service, and unapproved public endpoints are rejected. ]]>

T09 · Insecure Skill Coding Practices

Error
Location
scripts/diagnosis_sufficiency_review.py:268
Finding
Insufficient Data Minimization Before Medical Records Are Sent to the LLM<![CDATA[ ## Vulnerability Details **File Location**: `scripts/diagnosis_sufficiency_review.py:268-303, 399-436, 486-510` **Vulnerability Type**: Excessive disclosure of sensitive medical information **Risk Level**: High ### Vulnerable Code When no preferred document is selected, the implementation concatenates every nonempty document: ```python fallback_chunks: list[str] = [] for index, doc in enumerate(docs): if not isinstance(doc, dict): continue title = str(doc.get("fileName") or doc.get("docName") or f"文书#{index}") doc_type = str(doc.get("docClassName") or "未标注文书") content = str(doc.get("content", "")).strip() if content: fallback_chunks.append(f"【文书#{index} / {doc_type} / {title}】\n{content}") if not fallback_chunks: return [] return [ { "doc_index": -1, "doc_type": "fallback_concat", "file_name": "全部文书拼接", "content": truncate_text_by_utf8_bytes("\n\n".join(fallback_chunks)), } ] ``` The selected content is inserted into the LLM prompt: ```python 【病例文书】 {_evidence_text(evidence_docs) or '未检索到优先文书'} ``` The full prompt is then transmitted: ```python payload = { "model": model_config.model_id or selected_model, "messages": [{"role": "user", "content": prompt}], "temperature": model_config.temperature, } ``` ### Technical Analysis The Skill documentation tells callers to de-identify records, but the implementation does not enforce that requirement. There is no local detection or redaction of names, government identifiers, telephone numbers, addresses, record identifiers, or unrelated clinical details before transmission. The preferred-document logic can select up to eight document bodies. If the expected document types do not match, the fallback path combines all nonempty documents into one prompt, subject only to a 24,000-byte truncation limit. A byte limit controls request size but does not provide privacy protection or ensure that only evidence necessary f ...[truncated 1629 chars]
Remediation
<![CDATA[ ## Remediation Suggestions 1. Implement local de-identification before any network request: - Remove or tokenize names, patient numbers, identity numbers, telephone numbers, email addresses, and detailed addresses. - Strip metadata fields that are not required for the decision. 2. Fail closed when likely direct identifiers are detected and cannot be reliably redacted. 3. Replace whole-document transmission with local evidence extraction: - Select only passages relevant to the candidate diagnosis and guideline criteria. - Apply strict per-passage and total-request limits. - Avoid the current fallback that concatenates every document. 4. Require explicit opt-in for external LLM transmission and clearly identify what data will be sent. 5. Provide a genuinely offline mode that does not require an LLM credential and document its reduced capabilities. 6. Establish retention, access-control, and processing agreements for any approved model service. 7. Add privacy tests using synthetic identifiers to verify that sensitive fields never appear in outbound request bodies. ]]>

other

Warning
Location
scripts/diagnosis_sufficiency_review.py:399
Finding
Untrusted Medical-Record Text Can Manipulate the LLM Audit Decision<![CDATA[ ## Vulnerability Details **File Location**: `scripts/diagnosis_sufficiency_review.py:399-436, 582-601` **Vulnerability Type**: Indirect prompt injection **Risk Level**: Medium ### Vulnerable Code Untrusted record content is placed directly into the model prompt: ```python def _build_prompt( *, case_id: str, diagnosis: CaseDiagnosis, guideline: SufficiencyGuideline, evidence_docs: list[dict[str, Any]], ) -> str: role_label = "主诊断" if _scope(diagnosis.role) == "primary" else "其他诊断" prompt=f""" 你是住院病历"诊断依据充分性审核助手"。 请严格根据【审核指南】、【待审核病例信息】和【病例文书】完成审核。不得脑补。 你的任务是判断给出诊断的"依据是否充分" ... 【审核指南】 {guideline.guideline_text} 【待审核病例信息】 待审核诊断名称:{diagnosis.name} 是否主诊断:{role_label} 【病例文书】 {_evidence_text(evidence_docs) or '未检索到优先文书'} """ return prompt ``` The model-generated decision is normalized and accepted as the review result: ```python raw_response = llm_client.complete( _build_prompt(case_id=case_id, diagnosis=diagnosis, guideline=guideline, evidence_docs=evidence_docs), model_name=model_name, ) parsed = parse_json_object(raw_response) thinking = _extract_audit_thinking(raw_response) if thinking: parsed["thinking"] = thinking review_payload = _normalize_review_payload(parsed) if not review_payload["reason"]: review_payload["reason"] = "模型已完成诊断依据充分性审核。" ``` ### Technical Analysis Medical-record content is untrusted input. It may contain natural-language instructions intentionally inserted by an attacker or accidentally copied from another source. The implementation interpolates that content into the same user message as the operational instructions and guideline. Textual delimiters such as `【病例文书】` do not create a security boundary for an LLM. A document can include instructions telling the model to ignore the preceding rules and return a chosen JSON decision. Because any syntactically valid model JSON is normalized and used without independent evidence verification, the injected instruction can affect ` ...[truncated 1479 chars]
Remediation
<![CDATA[ ## Remediation Suggestions 1. Treat all record text as untrusted data and state this explicitly in a higher-priority system message. 2. Use separate structured message roles where supported: - Put immutable audit policy in the system message. - Supply evidence as a clearly encoded data object rather than free-form instructions. 3. Detect and flag instruction-like phrases in evidence documents for human review. 4. Require the model to return structured evidence references, including document index and exact quotation. 5. Independently verify that every cited quotation exists in the supplied document and that the decision satisfies deterministic guideline checks. 6. Default to `pending manual review` when citations are absent, fabricated, contradictory, or contain instruction-like content. 7. Avoid using the model result as the sole authority for consequential coding or reimbursement decisions. 8. Add adversarial tests containing prompt-injection instructions, fake JSON, role markers, and requests to ignore prior rules. ]]>
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • System Prompt LeakageDirect Leakage, Indirect Extraction, Tool-Based Exfiltration
  • Taint TrackingDirect Taint Flow, Variable-Mediated Taint Flow, Credential Exfiltration Chain
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
Findings (15)

Tainted flow: 'req' from os.getenv (line 509, credential/environment) → urllib.request.urlopen (network output)

Critical
Category
Data Flow
Content
headers["Authorization"] = f"Bearer {self._settings.api_key}"
        req = request.Request(url=url, headers=headers, method="GET")
        try:
            with request.urlopen(req, timeout=self._settings.timeout) as response:
                return json.loads(response.read().decode("utf-8"))
        except error.HTTPError as exc:
            if exc.code == 404:
Confidence
90% confidence
Finding
This code performs outbound HTTP GETs to a URL derived from environment configuration, so a compromised or misconfigured environment can cause the service to contact unintended hosts and disclose diagnosis codes, scope values, and bearer credentials. The data here is less sensitive than full case text, but the pattern still creates trust-boundary and potential SSRF/exfiltration concerns.

Tainted flow: 'req' from os.getenv (line 509, credential/environment) → urllib.request.urlopen (network output)

Critical
Category
Data Flow
Content
headers={"Content-Type": "application/json", **{key: value for key, value in headers.items() if value}},
            method="POST",
        )
        opener = request.urlopen(req) if not self._settings.timeout else request.urlopen(req, timeout=self._settings.timeout)
        with opener as response:
            response_payload = json.loads(response.read().decode("utf-8"))
        choices = response_payload.get("choices") or []
Confidence
98% confidence
Finding
The LLM request uses caller-controlled/base-configured network destinations and includes sensitive medical-record evidence in the POST body, so secrets and patient data can be transmitted off-host. Even if the appkey itself is intended for outbound use, the dangerous part is that untrusted configuration can redirect protected health information to arbitrary endpoints, creating data-exfiltration/SSRF-like risk.

Ae1

High
Category
analysis-evasion
Content
- 发布目录只保留 `SKILL.md`、`_meta.json`、`scripts/`;示例输入、运行输出、自测脚本放在 skill 包外。
Confidence
100% confidence
Finding
Referenced artifact was not completely inspected

Direct Prompt Extraction

High
Category
System Prompt Leakage
Content
def load_guideline_api_settings_from_env() -> GuidelineApiSettings:
    return GuidelineApiSettings(
        base_url=_required_env("GUIDELINE_API_BASE"),
        api_key=os.getenv("GUIDELINE_API_KEY", ""),
        timeout=_env_int("GUIDELINE_API_TIMEOUT", 30),
Confidence
85% confidence
Finding
Skill contains instructions that could directly expose system prompts, internal rules, or hidden instructions to users or external parties.

Direct Prompt Extraction

High
Category
System Prompt Leakage
Content
for candidate in diagnosis_code_lookup_candidates(diagnosis.code):
        guideline = repository.find_diagnosis_sufficiency_guideline_by_code_scope(candidate, scope)
        if guideline is not None:
            return guideline
    return None
Confidence
85% confidence
Finding
Skill contains instructions that could directly expose system prompts, internal rules, or hidden instructions to users or external parties.

Missing User Warnings

High
Confidence
97% confidence
Finding
The prompt construction and chat POST include raw medical-document content and ask the remote model to process it, without any explicit warning, confirmation, or consent mechanism. Because the skill handles healthcare case records, silent transmission of full document text materially increases privacy, compliance, and data-governance risk.

Direct Prompt Extraction

High
Category
System Prompt Leakage
Content
【病例文书】
{_evidence_text(evidence_docs) or '未检索到优先文书'}
    """
    return prompt


def _extract_audit_thinking(text: str) -> str:
Confidence
85% confidence
Finding
Skill contains instructions that could directly expose system prompts, internal rules, or hidden instructions to users or external parties.

Description-Behavior Mismatch

High
Confidence
99% confidence
Finding
The skill’s stated purpose is diagnosis-sufficiency review, but it sends case-document contents to an external LLM service during processing. In a medical context this is highly sensitive because records can contain PHI/PII, and the transfer is not obvious from the interface contract.

Context-Inappropriate Capability

High
Confidence
99% confidence
Finding
The CLI/payload lets the caller provide an arbitrary LLM base URL, which means the skill can be induced to send complete case evidence and credentials to attacker-controlled infrastructure. In this healthcare setting, that turns normal model invocation into a straightforward exfiltration channel for protected patient data.

Lp3

Medium
Category
MCP Least Privilege
Confidence
90% confidence
Finding
The skill advertises capabilities that include environment access, file read/write, and network use, but it does not declare any explicit tool scope or permission boundaries. In a medical-review skill that handles sensitive records and user-supplied credentials like appkey, this increases the risk of unintended data exfiltration, unsafe file writes, or broader-than-expected runtime behavior if the implementation is invoked in a permissive agent environment.

Natural-Language Policy Violations

Medium
Confidence
91% confidence
Finding
The skill hardcodes Chinese document types, Chinese prompt instructions, and Chinese output labels, which effectively constrain the interaction to a specific language/locale. There is no visible mechanism offering the user a language choice or documenting this as an explicitly region-scoped tool.

Ssd 3

Medium
Confidence
96% confidence
Finding
The prompt embeds unfiltered case-document text and instructs the model to return corresponding original case text in the output source field, increasing the chance of reproducing and propagating sensitive PHI. In a medical-review skill, this expands exposure beyond processing into downstream logging, storage, and display of raw record content.

Missing User Warnings

Medium
Confidence
88% confidence
Finding
The script can write JSON review output derived from medical records to an arbitrary filesystem path, which may persist sensitive diagnosis-related data outside controlled storage. Although output persistence is often expected in CLI tools, in this healthcare context the results are themselves sensitive and the code provides no safeguards such as redaction, permission hardening, or user warning about PHI retention.

Missing User Warnings

Medium
Confidence
94% confidence
Finding
The --save-prepared path writes preprocessed medical record text to disk, and the prepared text includes document contents that may contain highly sensitive PHI/PII. In a medical-review skill, this creates a real confidentiality risk because debug artifacts can persist on shared hosts, backups, or developer workstations without encryption, access controls, or explicit retention handling.

Natural-Language Policy Violations

Low
Confidence
87% confidence
Finding
The natural-language instructions, examples, and operational guidance are entirely presented in Chinese, which effectively forces a specific language for users and integrators. Under the policy, language constraints should either be optional for the user or clearly justified as region-specific; this file does neither.

Static analysis

No suspicious patterns detected.