T09 · Insecure Skill Coding Practices
Error
- Location
- src/llm-analyzer.ts:31
- Finding
- Webhook-Derived Text Can Manipulate Autonomous Financial Decisions Through Prompt Injection<![CDATA[ ## Vulnerability Details **File Location**: `src/index-helpers.ts:62-64`, `src/llm-analyzer.ts:31-59`, `src/index.ts:61-66` **Vulnerability Type**: Indirect prompt injection leading to unauthorized autonomous actions **Risk Level**: High ### Vulnerable Code `src/index-helpers.ts:62-64` extracts the cancellation reason directly from the webhook: ```ts const cancelReason = (obj.cancel_reason as string) ?? (obj.cancelReason as string) ?? "not provided"; ``` `src/llm-analyzer.ts:31-59` interpolates that untrusted value directly into the model prompt: ```ts export function buildChurnPrompt(ctx: ChurnContext): string { return `You are a SaaS retention analyst. A customer is about to churn. Analyze and recommend ONE action. Customer: ${ctx.customerEmail} Plan: ${ctx.productName} ($${ctx.price}/mo) Tenure: ${ctx.tenureMonths} months Total Revenue: $${ctx.totalRevenue} Cancel Reason: ${ctx.cancelReason || "not provided"} Available actions (pick exactly one): - CREATE_DISCOUNT: Create a retention discount. Params: { percentage: 10-50, durationMonths: 1-6 } - SUGGEST_PAUSE: Pause subscription instead of cancel. Params: {} - NO_ACTION: Let the customer go. Params: {} Rules: - High-value customers (>$500 total or >6 months): prefer CREATE_DISCOUNT with 20-40% - Low-tenure (<2 months) or low-value: prefer NO_ACTION - Medium cases: consider SUGGEST_PAUSE - Include confidence (0-1) in your assessment Respond in JSON only: {"action": "CREATE_DISCOUNT|SUGGEST_PAUSE|NO_ACTION", "reason": "one sentence", "confidence": 0.0-1.0, "params": {...}}`; } ``` `src/index.ts:61-66` treats model confidence as authorization for execution: ```ts // Auto-execute if confidence is high enough if (shouldAutoExecute(decision, AUTO_EXECUTE_THRESHOLD)) { const result = await executeAction(decision, churnCtx, creem as any); const resultMsg = formatActionResult(result, churnCtx); await bot.sendMessage(`🤖 Auto-executed (confidence ${Math.round(decision.confidence * 100)} ...[truncated 2345 chars]
- Remediation
- <![CDATA[ ## Remediation Suggestions 1. Treat all webhook and customer fields as untrusted data, including cancellation reasons, product names, and customer identifiers. 2. Place untrusted fields in a clearly delimited data section and explicitly state that their contents must never be interpreted as instructions. 3. Prefer structured model input or tool schemas rather than concatenating external text into a policy prompt. 4. Do not use model-generated confidence as an authorization mechanism. 5. Require explicit human approval for every state-changing financial action, or restrict automatic execution to deterministic, locally evaluated rules. 6. Derive action type and permitted parameters from server-side policy after the model response. 7. Add adversarial tests containing cancellation reasons such as instruction overrides, fake JSON responses, and requests for high-confidence actions. 8. Record an audit event containing the source webhook ID, chosen policy, approver, and executed Creem operation. ]]>
