T09 · Insecure Skill Coding Practices
Error
- Location
scripts/compression_evaluator.py:24- Finding
Unredacted Conversation Data Transmitted to a Third-Party Evaluation API
- Content
View full analysis
dict: r = requests.post( f"{_API_BASE}/pilot", headers={"Authorization": f"Bearer {SKILLBOSS_API_KEY}", "Content-Type": "application/json"}, json=body, timeout=60, ) return r.json() ``` ```python judge_input = json.dumps({ "probe_question": probe.question, "model_response": response, "compacted_context": context[:2000], # trim for token budget "ground_truth": probe.ground_truth or "", "rubric_criterion": criterion["id"], "criterion_question": criterion["question"], }) try: result = _pilot({ "type": "chat", "inputs": { "messages": [ {"role": "system", "content": JUDGE_SYSTEM_PROMPT}, {"role": "user", "content": ( f"Evaluate the following and return JSON with keys " f"'score' (0-5 float) and 'reasoning' (string):\n{judge_input}" )} ] }, "prefer": "balanced" }) ``` ### Technical Analysis The evaluator serializes the model response, extracted ground truth, probe question, and up to 2,000 characters of compressed conversation context into an LLM judge request. `_pilot()` then sends that request to the fixed third-party endpoint `https://api.heybossai.com/v1/pilot`. Conversation context used for software-development evaluation may contain proprietary code fragments, internal file paths, error messages, architectural decisions, customer information, credentials, access tokens, or other secrets. Limiting context to 2,000 characters controls payload size but does no ...[truncated 2790 chars]- Remediation
View remediation
str: key = os.getenv("SKILLBOSS_API_KEY") if not key: raise RuntimeError("Remote evaluation requires SKILLBOSS_API_KEY") return key ``` - Only access the credential after the caller explicitly selects remote evaluation. 6. **Make the endpoint controllable and auditable** - Permit administrators to configure an approved HTTPS endpoint. - Validate the URL against an allowlist and prohibit insecure HTTP. - Document the service operator and data-processing expectations. 7. **Improve request handling** - Call `raise_for_status()` before parsing the response. - Catch narrow exception types rather than suppressing all exceptions. - ...[truncated 650 chars]