T09 · Insecure Skill Coding Practices
Warning
- Location
references/audit-script-template.sh:31- Finding
Blocking Audit Failures Return a Successful Exit Status
- Content
View full analysis
/dev/null || echo 0) if [ "$count" -lt "$MIN_PMID_PER_FILE" ]; then echo " ❌ $f: only $count citations" fail=$((fail+1)) else echo " ✅ $f: $count citations" fi done [ $fail -eq 0 ] && echo " → PASS ✅" fi ``` The script concludes as follows: ```bash echo "" echo "================================================" echo " Audit complete. Review ⚠️ warnings and ❌ failures." echo " ❌ blocks publishing. ⚠️ documented in errata." echo "================================================" ``` The same pattern is repeated across the citation-duplicate, anti-sycophancy, and H1 checks: failures are counted and displayed, but they are not propagated to the process exit status. ### Technical Analysis The audit script states that critical failures block publication, but it does not maintain a global blocking-failure status or execute `exit 1` when a blocking check fails. Each section either resets its local `fail` variable or merely prints a warning. The final command is a successful `echo`. Therefore, under normal execution, the script returns exit status zero even when it has printed one or more critical failure messages. This creates a mismatch between human-readable output and machine-readable status. CI systems, publication scripts, and AI agents commonly determine whether a quality gate passed by checking only the command exit status. Such consumers will interpret the audit as successful. Some checks, including length, gender balance, and safety-keyword coverage, do not increment a blocking counter at all. They therefore cann ...[truncated 1511 chars]- Remediation
View remediation
0 )); then printf 'Audit failed with %d blocking issue(s).\n' "$blocking_failures" >&2 exit 1 fi printf 'Audit passed with %d warning(s).\n' "$warnings" exit 0 ``` 5. Decide explicitly whether length and safety-keyword failures are blocking. Their implementation should match the policy stated in `SKILL.md`. 6. Add automated tests that invoke the script and assert: - Compliant input returns status 0. - Missing citations return a nonzero status. - Duplicate citations return a nonzero status. - Missing required H1 headings return a nonzero status. - Missing required critical reviews return a nonzero status. ]]>