T09 · Insecure Skill Coding Practices
Warning
- Location
scripts/propose.py:268- Finding
Undisclosed transmission of target and evaluation data to an external model CLI
- Content
View full analysis
dict | None: """Use LLM to propose a SKILL.md fix based on evaluator failures. Reads the current SKILL.md, sends it + failure details to claude -p, and asks for a targeted fix. """ import shutil import subprocess import json as _json skill_md = target / "SKILL.md" if target.is_dir() else target if not skill_md.exists(): return None if shutil.which("claude") is None: return None skill_content = skill_md.read_text(encoding="utf-8") failures_text = "\n".join( f"- Task '{t['task_id']}': {t.get('details', t.get('error', 'unknown'))}" for t in failed_tasks ) prompt = ( "You are improving a SKILL.md file based on evaluator task failures.\n\n" f"## Current SKILL.md\n```\n{skill_content[:3000]}\n```\n\n" f"## Failed Tasks\n{failures_text}\n\n" "Analyze why these tasks failed and propose a TARGETED change to SKILL.md " "that would fix the failures without breaking what already works.\n\n" "Respond with ONLY a JSON object:\n" '{{"section_heading": "## ", ' '"action": "append_markdown_section" or "replace_markdown_section", ' '"content_lines": ["line1", "line2", ...], ' '"rationale": "why this fixes the failures"}}' ) try: result = subprocess.run( ["claude", "-p", "--output-format", "json"], input=prompt, capture_output=True, text=True, timeout=120, ) ``` The default analysis path performs the same operation: ```python skill_content = skill_md.read_text(encoding="utf-8") prompt = ( "You are a skill quality analyst. Analyze this S ...[truncated 3180 chars]- Remediation
View remediation