T09 · Insecure Skill Coding Practices
Error
- Location
- scripts/analyzer.py:265
- Finding
- Broken Score Normalization Produces Unreliable Risk Classifications<![CDATA[ ## Vulnerability Details **File Location**: `scripts/analyzer.py`, lines 265-286 **Vulnerability Type**: Incorrect security-score normalization and fail-open risk classification **Risk Level**: High ### Vulnerable Code ```python def _calculate_scores(self) -> Dict[str, float]: """Calculate weighted scores.""" scores = {} for dimension, config in self.DIMENSIONS.items(): score = self.results.get(dimension, 0) scores[dimension] = score * config["weight"] * 10 return scores def _weighted_average(self, scores: Dict[str, float]) -> float: """Calculate weighted average.""" return sum(scores.values()) def _assess_risk(self, scores: Dict[str, float]) -> str: """Assess overall risk level.""" overall = sum(scores.values()) if overall >= 7.5: return "LOW" elif overall >= 5: return "MEDIUM" else: return "HIGH" ``` ### Technical Analysis Each raw dimension score is already expressed on a scale from 0 to 10. The calculation multiplies that score by its configured weight and then multiplies it by 10 again: ```python score * config["weight"] * 10 ``` Because the dimension weights sum to 1, this produces an aggregate range of 0 to 100. However, `_assess_risk()` applies thresholds designed for a range of 0 to 10. Consequently, an aggregate score only needs to reach 7.5 out of 100 to be classified as `LOW` risk. The same incorrectly scaled values are exposed as dimension scores and printed as though each were out of 10. For example, the functionality dimension has a weight of 0.25 and can produce a displayed value as high as 25. This is security-relevant because the tool is explicitly presented as a security review mechanism for third-party skills. A severely unsafe skill can receive an authoritative-looking `LOW` risk result when modest points from non-security dimensions overwhelm the incorrectly scaled threshold. ### Attack Path 1. An attacker prepares a malicious or unsafe ...[truncated 1249 chars]
- Remediation
- <![CDATA[ ## Remediation Suggestions 1. Preserve the intended 0-10 aggregate scale by removing the extra multiplication: ```python scores[dimension] = score * config["weight"] ``` 2. Keep the existing `7.5` and `5.0` thresholds only after confirming that the aggregate cannot exceed 10. 3. Report raw dimension scores separately from weighted contributions. For example: ```python raw_scores[dimension] = score weighted_total += score * config["weight"] ``` 4. Add explicit invariants before classification: ```python if not 0 <= overall <= 10: raise ValueError(f"Invalid normalized score: {overall}") ``` 5. Avoid allowing strong documentation or usability scores to conceal critical security findings. Introduce fail-closed rules, such as: - Any confirmed critical behavior results in `HIGH` risk. - A security score below a defined minimum prevents `LOW` risk. - Incomplete scan coverage produces `UNKNOWN` or `REVIEW REQUIRED`, not `LOW`. 6. Add unit tests for minimum, maximum, and boundary values, including: - All dimensions equal to zero. - All dimensions equal to ten. - Security equal to zero with high non-security scores. - Aggregates immediately below and above 5.0 and 7.5. ]]>