T01 · Skill Instruction Hijacking
Error
- Location
- curriculum.py:145
- Finding
- Untrusted User Intent Is Converted into Instructions for an Autonomous Agent<![CDATA[ ## Vulnerability Details **File Location**: `curriculum.py:145-168`; `trainer.py:304-335`; `main.py:63-70, 136-139` **Vulnerability Type**: Indirect prompt injection and unsafe instruction forwarding **Risk Level**: High ### Vulnerable Code ```python async def design_curriculum(llm: LLMHandler, user_intent: str) -> Curriculum: """Use the LLM to generate a training curriculum from a user intent string.""" prompt = CURRICULUM_DESIGN_PROMPT.format( stage_min=config.STAGE_COUNT_MIN, stage_max=config.STAGE_COUNT_MAX, tasks_min=config.TASKS_PER_STAGE_MIN, tasks_max=config.TASKS_PER_STAGE_MAX, ) conv = Conversation(system_prompt=prompt) conv.add( "user", f"Please design a training curriculum for the following requirement:\n\n" f"{user_intent}", ) logger.info("Generating training curriculum...") data = await llm.chat_json(conv, temperature=config.CURRICULUM_TEMPERATURE) return Curriculum.from_dict(data) ``` ```python if attempt == 1: instruction = ( f"Present the following training task to the Claw agent. " f"Craft a clear, well-structured message that the agent " f"will receive directly.\n\n" f"Task ID: {task.task_id}\n" f"Description: {task.description}\n" f"Scenario:\n{task.scenario}\n" f"Expected behavior: {task.expected_behavior}" ) else: instruction = ( f"The agent's previous response did not fully meet the " f"standards. Here is the evaluation:\n\n" f"Score: {result.score}/10\n" f"Weaknesses: {', '.join(result.weaknesses)}\n" f"Feedback: {result.feedback}\n" f"Suggestion: {result.suggestion}\n\n" f"Generate a follow-up message to help the agent improve. " f"You may rephrase the task, provide hints, or break it " f"into smaller steps — whatever you think will be most " f"effective as a trainer." ) trainer_msg ...[truncated 2436 chars]
- Remediation
- <![CDATA[ ## Remediation Suggestions 1. Display every generated task, scenario, expected behavior, evaluation criterion, and exact outbound agent message before transmission. 2. Require explicit approval for each exact outbound message rather than approving only a curriculum summary. 3. Enforce a semantic policy that rejects tasks involving credentials, persistence, system configuration, destructive operations, unauthorized communications, or unrelated objectives. 4. Define an allowlisted task schema with bounded field lengths and permitted capability categories. 5. Run the target agent with a restricted tool profile during training, disabling command execution, sensitive filesystem access, credential access, and external messaging unless specifically required. 6. Treat trainer output as an untrusted proposal, not an executable instruction. 7. Record provenance linking every outbound instruction to the approved user request and generated curriculum field. ]]>
