T01 · Skill Instruction Hijacking
Error
- Location
- scripts/analyze.py:292
- Finding
- Indirect prompt injection through untrusted search content<![CDATA[ ## Vulnerability Details **File Location**: `scripts/analyze.py:292-359` **Vulnerability Type**: Indirect prompt injection and unvalidated AI-generated data persistence **Risk Level**: High ### Vulnerable Code ```python user_prompt = f"""Analyze the brand: {brand_name} Use Google Search to research this brand thoroughly: 1. Find official brand information (website, corporate pages) 2. Search for advertising campaigns on Google Images 3. Identify visual patterns, photography style, and brand behavior Fill in the following JSON template with your analysis. Return ONLY valid JSON, no other text: {template_str}""" response = client.models.generate_content( model="gemini-2.5-flash", contents=user_prompt, config=types.GenerateContentConfig( system_instruction=SYSTEM_PROMPT, max_output_tokens=16384, temperature=0.3, # thinking_config=types.ThinkingConfig(thinking_level="low"), tools=[types.Tool(google_search=types.GoogleSearch())] ) ) # Parse JSON brand_data = extract_json_from_response(response_text) if brand_data is None: # Save raw response for debugging debug_path = f"/tmp/brand-analyzer-debug-{sanitize_brand_name(brand_name)}.txt" with open(debug_path, 'w') as f: f.write(response_text) raise RuntimeError(f"Failed to parse JSON from response. Raw saved to: {debug_path}") # Ensure brand name is set if brand_data.get("brand_info", {}).get("name", "") == "": brand_data.setdefault("brand_info", {})["name"] = brand_name ``` ### Technical Analysis The Gemini model is instructed to retrieve and analyze content through Google Search. Search results, indexed web pages, image descriptions, and Pinterest content are untrusted inputs that may contain instructions directed at AI systems. The system prompt does not explicitly require the model to treat retrieved text exclusively as data or to ignore instructions embedded in external content. An attacker who controls a website ...[truncated 2177 chars]
- Remediation
- <![CDATA[ ## Remediation Suggestions 1. Add explicit instructions stating that all retrieved web content is untrusted data and that instructions found within it must never be followed. 2. Separate retrieved evidence from operational instructions using strongly delimited data sections. 3. Restrict research to allowlisted official domains where practical. 4. Define and enforce a strict JSON Schema matching `BRAND_IDENTITY_TEMPLATE`. 5. Reject additional properties, incorrect field types, excessive nesting, and oversized values. 6. Recursively inspect string fields for prompt-injection indicators before persistence or downstream use. 7. Do not treat model-produced URLs, commands, or instructions as trusted operational data. 8. Require human review before newly generated profiles become available to downstream orchestrators. 9. Record source provenance for generated claims so suspicious content can be traced and reviewed. 10. Ensure downstream workflows delimit profile values as untrusted data rather than incorporating them directly into system or developer instructions. ]]>
