Advanced ML Classification Skill

自动化生成工业级机器学习分类算法代码、调用算法做预测、输出准确率对比和可视化结果，支持新手友好的结果解读。

Security Scan

The skill appears to implement the stated ML benchmarking purpose, but it executes code generated by an external LLM (Codex/GPT) and uses an OpenAI API key (not declared in metadata), creating a meaningful risk of executing arbitrary code or leaking dataset content.

Purpose & Capability

The name/description (automated ML classification benchmarking) matches the included code, example datasets, and UI. Requiring scikit-learn, xgboost, lightgbm, plotly and streamlit (via requirements.txt) is proportionate. However the skill optionally uses OpenAI/Codex to generate algorithm code and GPT to produce plain-language interpretation — this external LLM dependency is not declared in the registry metadata (no required env vars). Using an LLM to generate training code is a plausible design choice for this purpose, but it should be explicitly documented as a required/optional capability.

Instruction Scope

The runtime executes training functions that may be generated by an external model (code-davinci-002 / Codex) and then runs that generated Python. Executing arbitrary generated Python at runtime expands scope beyond purely local, deterministic ML steps and can lead to arbitrary file system and network access, plus potential exfiltration of dataset contents. SKILL.md explicitly instructs use of Codex and GPT for code/interpretation; the app UI also accepts an OpenAI API key. There is no explicit guidance or sandboxing for inspecting or restricting generated code before execution.

Install Mechanism

There is no remote download/install spec in the skill definition; all code files are bundled and the provided shell helpers create a local venv and pip-install the packages listed in requirements.txt (packages from PyPI). No external arbitrary archive downloads or unknown URLs are used. This is low-to-moderate install risk typical for Python projects.

Credentials

The code reads an OpenAI API key (OPENAI_API_KEY env or user-supplied via the Streamlit UI) to call OpenAI (Codex/GPT). The skill metadata declared no required environment variables or primary credential; that's a mismatch. An OpenAI key is a sensitive credential and should be declared as optional in metadata so users understand the credential will be used and (potentially) sent dataset-related prompts.

Persistence & Privilege

The skill is not always-enabled and does not request system-wide permissions in the manifest. It does not appear to modify other skills or global agent settings. The main privilege concern is runtime: executing externally generated code, but this is not a persistence/privilege escalation issue in the skill metadata itself.

What to consider before installing

What to consider before installing or running this skill: - The skill can optionally use your OpenAI key to generate Python training functions (Codex) and to create natural-language interpretations. That key is sensitive; the manifest did not declare it. Expect prompts and possibly dataset metadata to be sent to OpenAI if you provide the key. - The skill may execute Python code returned by the LLM. Generated code can be arbitrary and could read files, open network connections, or exfiltrate data. If you plan to run it, do so only in an isolated environment (container or VM) and with non-sensitive example data until you audit the generated code. - If you do not want any data leaving your environment, do NOT provide an OpenAI key and do not enable features that call external LLMs; instead let the skill fall back to its bundled local templates (the code indicates it has local templates as a fallback). - Before trusting results: inspect the generated code (the UI exposes generated_codes) and verify it only uses sklearn/XGBoost/LightGBM and does not perform I/O or network calls you don't expect. - Prefer running the demo in a disposable virtualenv/VM, review requirements.txt before pip installing, and avoid running with production or sensitive datasets unless you have audited the runtime behavior and are comfortable with the LLM calls. Confidence is medium because parts of advanced_ml_skill.py were truncated in the provided listing; reviewing the complete run()/code-generation/execution implementation would reduce uncertainty.

输入参数

• data_path: str（必填）CSV 数据集路径
• target_col: str（必填）预测目标列名
• algorithms: list[str]（可选）默认 [ "逻辑回归", "决策树", "随机森林", "XGBoost", "LightGBM" ]
• test_size: float（可选）默认 0.2
• random_state: int（可选）默认 42

输出结构

• accuracy_results: dict[str, float|None]
• interpretation: str
• generated_codes: dict[str, str]
• visualization_data: dict

关键流程

1. 自动预处理（缺失值、类别编码、数值标准化）
2. 按算法生成训练代码（优先 code-davinci-002，失败回退本地模板）
3. 执行算法代码并统计准确率（失败时返回具体错误）
4. 可选交叉验证（StratifiedKFold/KFold/RepeatedStratifiedKFold）
5. 可选参数搜索（GridSearchCV/RandomizedSearchCV）
6. 生成置换特征重要性排序（默认对最佳算法）
7. 生成新手友好中文解读（优先 gpt-3.5-turbo）
8. 输出可视化数据（柱状图/折线图）

运行示例

cd /Users/bamboo/skills/advanced-ml-classification-skill/scripts
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
python generate_complex_demo.py
python advanced_ml_skill.py --data-path ./demo_complex.csv --target-col target_label --enable-cv --enable-search
streamlit run app.py