T09 · Insecure Skill Coding Practices
- Location
data/agents.json:285- Finding
Bundled Agent Prompt Recommends Unsafe Evaluation of Model-Generated Python
- Content
View full analysis
dict: response = ollama.chat( model='phi3', messages=[ {'role': 'system', 'content': SYSTEM_PROMPT}, {'role': 'user', 'content': f"Column: '{column_name}'\nSamples:\n" + "\n".join(sample_rows)} ] ) result = json.loads(response['message']['content']) forbidden = ['import', 'exec', 'eval', 'os.', 'subprocess'] if not result['transformation'].startswith('lambda'): raise ValueError("Rejected: output must be a lambda function") if any(term in result['transformation'] for term in forbidden): raise ValueError("Rejected: forbidden term in lambda") return result def apply_fix_to_cluster(df: pd.DataFrame, column: str, fix: dict) -> pd.DataFrame: if fix['confidence_score'] < 0.75: df['validation_status'] = 'HUMAN_REVIEW' df['quarantine_reason'] = f"Low confidence: {fix['confidence_score']}" return df transform_fn = eval(fix['transformation']) df[column] = df[column].map(transform_fn) df['validation_status'] = 'AI_FIXED' df['ai_reasoning'] = fix['reasoning'] df['confidence_score'] = fix['confidence_score'] return df ``` ### Technical Analysis The bundled `AI Data Remediation Engineer` agent prompt presents `eval()` as a safe way to execute a Python lambda generated by a language model. The preceding validation only verifies that the string begins with `lambda` and does not contain five forbidden substrings. A substring denylist does not create a Python sandbox. A lambda expression can access dangerous functionality without directly containing the listed strings. Potential bypass techniques include: - Traversing Python object metadata and subclass hierarchies. - Accessing cal ...[truncated 2276 chars]- Remediation
View remediation
