T09 · Insecure Skill Coding Practices
Warning
- Location
- poq.js:7
- Finding
- Forgeable and Replayable Proof-of-Quality Result<![CDATA[ ## Vulnerability Details **File Location**: `poq.js`, lines 7-20 **Vulnerability Type**: Weak validation and proof generation logic **Risk Level**: Medium ### Vulnerable Code ```js function benchmark(skillContent) { const lines = skillContent.split('\n').length; const quality = lines > 10 && skillContent.includes('PoW') ? 98 : 80; return quality; } function powQuality(score, difficulty = 4) { let nonce = 0; while (true) { const hashInput = score + nonce; const hash = crypto.createHash('sha256').update(hashInput).digest('hex'); if (hash.startsWith('0'.repeat(difficulty))) return { hash, nonce, score }; nonce++; } } ``` ### Technical Analysis The benchmark does not execute a test suite or measure the claimed speed, accuracy, or security properties. It assigns a score of `98` solely when the input contains more than ten lines and includes the literal string `PoW`; all other inputs receive `80`. The generated proof hashes only the concatenated score and nonce: ```js const hashInput = score + nonce; ``` It does not bind the proof to the evaluated file contents, a cryptographic digest of the skill package, the benchmark evidence, the threshold, or the evaluator version. Therefore, all inputs receiving the same score share an identical proof search space. A previously generated proof can be reused for a different skill or after the evaluated skill has been modified. This behavior conflicts with the verification and security-benchmarking claims in `SKILL.md`, because the implementation neither performs the documented security evaluation nor supplies enough data to verify that a proof belongs to a specific artifact. ### Attack Path 1. An attacker creates an unsafe or low-quality skill containing more than ten lines. 2. The attacker inserts the literal text `PoW` anywhere in the input. 3. The attacker invokes `node poq.js <skill_path> 95`. 4. The superficial benchmark assigns the skill a score of `98`. 5. The script searc ...[truncated 962 chars]
- Remediation
- <![CDATA[ ## Remediation Suggestions 1. Replace the text-based heuristic with deterministic, documented test suites that measure the claimed quality and security properties. 2. Calculate a cryptographic digest over the canonical contents of every evaluated file in the skill package. 3. Bind the artifact digest, benchmark results, threshold, evaluator version, configuration, timestamp or validity period, and nonce into the proof input. 4. Use an unambiguous serialization format, such as canonical JSON, before hashing rather than directly concatenating values. 5. Emit a structured proof containing all verification inputs and implement a verifier that recomputes the artifact digest, benchmark results, and proof-of-work. 6. Digitally sign the structured result when evaluator provenance and resistance to unauthorized proof generation are required. 7. Reject modified artifacts when their current digest does not match the digest embedded in the proof. 8. Update `SKILL.md` so its benchmarking, JSON output, and verification claims accurately match the implemented behavior. ]]>