T09 · Insecure Skill Coding Practices
- Location
references/or-pipeline.md:36- Finding
Potential Disclosure of Customer Data to an External LLM Endpoint
- Content
View full analysis
Vulnerability Details
File Location:
references/or-pipeline.md, lines 36-38, 45, and 81-93
Vulnerability Type: External transmission of potentially unredacted personal and business data
Risk Level: HighVulnerable Documentation Snippets
markdown - Endpoint: `https://maas.devops.xiaohongshu.com/customerserviceinference-glm-5/v1/chat/completions` - Model: `GLM-5.2-FP8`, header `api-key: <your MaaS api-key>`markdown The sample carries: session key, group, industry, broad-definition lead label, and compressed conversation text.markdown Determine whether the AI customer service behavior in the following merchant private-message conversation matches each rubric. [Rubrics] {all entries: id + rubric + criteria} [Conversation] {compressed conversation text}Supporting evidence in
references/data-schema.md, lines 22 and 35-41, establishes that source records and conversations can contain stable visitor identifiers, full message histories, telephone numbers, WeChat identifiers, lead-component events, and system-side lead-generation flags.Technical Analysis
The documented pipeline directs operators to send sampled customer conversations to a configured HTTPS LLM endpoint. Each sample includes a session key, experiment group, industry, lead outcome, and compressed conversation text.
The described compression procedure truncates messages and conversations, but truncation is not anonymization. No mandatory processing stage removes telephone numbers, WeChat IDs, names, addresses, stable session identifiers, account identifiers, URLs, secrets, or other sensitive free-form content before transmission. The pipeline also does not require an approved data-processing agreement, retention controls, endpoint logging controls, or explicit authorization for transferring production conversations.
Because the analysis specifically identifies telephone and WeChat details as lead signals, such identifiers are reasonably likely to ...[truncated 2137 chars]
- Remediation
View remediation
Remediation Suggestions
-
Add mandatory redaction before sampling or transmission
- Remove telephone numbers, WeChat IDs, email addresses, names, physical addresses, account IDs, URLs containing identifiers, and credentials.
- Apply redaction to every role and structured event field, not only visitor messages.
- Replace sensitive values with typed placeholders such as
[PHONE]and[WECHAT_ID].
-
Eliminate linkable identifiers
- Do not include production session keys in prompts.
- Generate random, per-run identifiers that cannot be mapped back without a separately protected lookup table.
- Exclude experiment metadata and industry information unless each field is necessary for the classification task.
-
Implement an enforceable preflight privacy gate
- Scan every outbound prompt for phone-number patterns, WeChat identifiers, secrets, and internal IDs.
- Block submission when residual sensitive values are detected.
- Record only redaction counts and non-sensitive operational metadata in audit logs.
-
Minimize submitted content
- Send only the turns required to assess each rubric.
- Remove lead-event text and post-outcome messages when they are unnecessary.
- Prefer classification on conversations truncated before the first lead event, which also reduces outcome leakage.
-
Constrain the receiving service
- Use only an organization-approved inference environment with documented access controls.
- Disable request and response retention where supported.
- Define retention, deletion, geographic-processing, and subprocessor requirements contractually.
- Restrict the API credential to the required endpoint and operation, store it in an approved secret manager, and rotate it regularly.
-
Prefer local processing for sensitive data
- Run the classifier in an approved local or isolated environment when production conversations cannot be reliably anonymized.
- Use synthetic or explicitl ...[truncated 422 chars]
-