Back to skill

Security audit

Dobby Harness Self-improving Coding Skills

Security checks for vulnerabilities and agentic risk

Overview

This skill is mostly coherent, but it persists broad agent/session data to local plaintext files and gives broad automation guidance that should be reviewed before use.

Review this before installing in sensitive workspaces. Avoid storing secrets, tokens, full environment dumps, private code context, or personal data in the WAL, buffer, or logs unless you add restrictive file permissions, redaction, retention limits, and encryption. Treat CI/CD and PR-commenting examples as requiring explicit user approval and safe target environments.

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (4)

T09 · Insecure Skill Coding Practices

Warning
Location
memory/wal.js:78
Finding
WAL checksum verification always accepts modified entries<![CDATA[ ## Vulnerability Details **File Location**: `memory/wal.js:78-83` **Vulnerability Type**: Broken integrity verification **Risk Level**: Medium ### Vulnerable Code ```javascript verifyChecksum() { if (!this.checksum) return true; const expected = this.calculateChecksum(); return this.checksum === expected; } ``` ### Technical Analysis `verifyChecksum()` invokes `calculateChecksum()`, which calculates a new checksum and assigns it directly to `this.checksum`. The subsequent comparison therefore compares the newly calculated value with itself rather than comparing it with the checksum originally read from disk. Entries with no checksum are also accepted because the method immediately returns `true` when `this.checksum` is absent. As a result, an attacker who can modify a WAL file can alter its transaction ID, event data, metadata, sequence, or timestamp without the modification being detected by `readAllEntries()`. This is not a cryptographic authenticity control and does not protect recovery data against malicious local modification. ### Attack Path 1. The attacker obtains write access to the configured WAL directory, such as through the same operating-system account, an overly permissive directory, or another compromised local process. 2. The attacker edits a `wal-*.log` entry and changes its transaction metadata or event data. 3. The attacker supplies an arbitrary checksum or removes the checksum. 4. `readAllEntries()` parses the modified entry and calls `verifyChecksum()`. 5. Verification recalculates and overwrites the stored checksum, or accepts the missing checksum. 6. `recover()` processes the modified entry as valid recovery state. ### Impact Assessment Successful exploitation can compromise the integrity of persisted task and transaction state. An attacker may falsify transaction completion, alter recovered metadata, hide corruption, or cause the application to recover misleading state. Exploitation requires local write access t ...[truncated 218 chars]
Remediation
<![CDATA[ ## Remediation Suggestions - Preserve the checksum loaded from disk in a separate variable before calculating the expected value. - Make checksum calculation a pure function that does not mutate the entry. - Reject missing checksums whenever `enableChecksum` is enabled. - Use a timing-safe comparison for cryptographic authentication values. - If protection from malicious modification is required, replace the current non-cryptographic hash with HMAC-SHA-256 using a protected key. - Validate entry types, sequence values, transaction identifiers, timestamps, and data schemas before recovery. - Add tests proving that changes to every protected field cause verification to fail. Example: ```javascript calculateChecksumValue() { const content = JSON.stringify({ type: this.type, transactionId: this.transactionId, data: this.data, sequence: this.sequence, timestamp: this.timestamp, }); return createHash('sha256').update(content).digest('hex'); } verifyChecksum() { if (!this.checksum) return false; const stored = this.checksum; const expected = this.calculateChecksumValue(); return stored === expected; } ``` For authenticity rather than accidental-corruption detection, use an HMAC instead of an unkeyed hash. ]]>

T09 · Insecure Skill Coding Practices

Warning
Location
memory/wal.js:321
Finding
WAL persists unrestricted task data in plaintext files<![CDATA[ ## Vulnerability Details **File Location**: `memory/wal.js:321-328` **Vulnerability Type**: Plaintext persistence of potentially sensitive agent state **Risk Level**: Medium ### Vulnerable Code ```javascript flush() { if (this.buffer.length === 0) return; appendFileSync(this.currentFile, this.buffer.join('')); this.buffer = []; this.bufferSize = 0; } ``` The flushed buffer contains serialized `LogEntry` objects populated by caller-supplied transaction metadata and event data. ### Technical Analysis The WAL API accepts arbitrary metadata and event data, serializes the resulting entries as JSON, and appends them to plaintext files. The write operation does not specify a restrictive file mode, and the containing directory is also created without an explicit mode. Effective access therefore depends on the process umask and surrounding filesystem permissions. The implementation does not redact secret-bearing fields, encrypt sensitive values, enforce a storage schema, or impose a meaningful limit on the total retained data. Compaction also creates another plaintext copy, while the intended backup movement is not implemented. Task inputs, intermediate outputs, error details, tokens accidentally included in context, or other private agent state may consequently remain accessible on disk. ### Attack Path 1. An application passes sensitive task metadata, intermediate results, errors, or contextual data to `begin()` or `log()`. 2. The data is serialized into a `LogEntry` and placed in the WAL buffer. 3. `flush()` appends the serialized entry to a plaintext `wal-*.log` file. 4. A local user or process with read access to the WAL directory reads the retained content. 5. The exposed information may be used outside the Skill, depending on what the integrating application stored. ### Impact Assessment The potential impact is disclosure of task descriptions, user data, intermediate results, operational metadata, or credentials accidentally include ...[truncated 405 chars]
Remediation
<![CDATA[ ## Remediation Suggestions - Create WAL directories with mode `0700` and WAL files with mode `0600`. - Open files explicitly with secure flags and modes rather than relying on the host umask. - Restrict the WAL directory to an application-controlled location and reject unsafe or unexpected paths. - Define schemas for permitted metadata and event data. - Recursively redact passwords, tokens, API keys, authorization headers, cookies, and private keys before serialization. - Enforce per-entry, per-transaction, segment, and total-retention limits. - Use authenticated encryption for sensitive state, with keys obtained from a protected secret store. - Implement retention and secure-cleanup policies for segments, compacted files, and backups. - Clearly document that callers must not persist secrets unless encryption is enabled. ]]>

T09 · Insecure Skill Coding Practices

Warning
Location
memory/working-buffer.js:244
Finding
Working Buffer stores arbitrary session state in an unhardened plaintext JSON file<![CDATA[ ## Vulnerability Details **File Location**: `memory/working-buffer.js:244-259` **Vulnerability Type**: Plaintext persistence without access-control hardening or size restrictions **Risk Level**: Medium ### Vulnerable Code ```javascript async save() { const data = { version: 1, timestamp: Date.now(), count: this.entries.size, entries: {}, }; for (const [key, entry] of this.entries) { data.entries[key] = entry.toJSON(); } const filePath = join(this.bufferDir, 'buffer.json'); const content = JSON.stringify(data, null, 2); writeFileSync(filePath, content); return { saved: true, count: this.entries.size }; } ``` ### Technical Analysis `WorkingBuffer.set()` accepts arbitrary values and metadata. `save()` serializes every in-memory entry into a single plaintext `buffer.json` file. The file is written without an explicit restrictive mode, encryption, redaction, schema validation, or payload-size limit. Automatic saving is enabled by default, so sensitive state can be written immediately and again by the periodic save timer without a separate persistence decision from the caller. The documented session-state model includes user identity, workspace paths, environment variables, task history, intermediate results, and reasoning-related data, all of which may be sensitive. Because all entries are serialized into one file, a single oversized or deeply nested value may also cause significant CPU, memory, or disk use during repeated saves. ### Attack Path 1. Sensitive or attacker-controlled content is supplied to `WorkingBuffer.set()` or `WorkingBuffer.import()`. 2. Default automatic saving invokes `save()`. 3. All entries are serialized into plaintext `buffer.json`. 4. The file inherits permissions derived from the runtime environment rather than enforcing owner-only access. 5. Another local process with filesystem access reads the state, or oversized content repeatedly consumes memory and disk during automatic sa ...[truncated 530 chars]
Remediation
<![CDATA[ ## Remediation Suggestions - Create the buffer directory with mode `0700` and `buffer.json` with mode `0600`. - Write to a securely created temporary file in the same directory, flush it, and atomically rename it to prevent partial-state corruption. - Reject symlinks and ensure the resolved destination remains inside an approved application data directory. - Introduce allowlisted schemas for keys, values, and metadata. - Add maximum entry count, key length, value size, nesting depth, and total serialized-size limits. - Redact known secret fields before persistence. - Encrypt sensitive state with authenticated encryption and store keys separately. - Make automatic persistence opt-in for sensitive applications. - Provide explicit retention, deletion, and shutdown methods, and ensure timers are stopped when the buffer is no longer needed. ]]>

T09 · Insecure Skill Coding Practices

Note
Location
harness/utils/logger.js:47
Finding
File logger writes unrestricted objects without secret redaction or secure file controls<![CDATA[ ## Vulnerability Details **File Location**: `harness/utils/logger.js:47-87` **Vulnerability Type**: Sensitive-data exposure and log injection **Risk Level**: Low ### Vulnerable Code ```javascript format(level, levelName, ...args) { if (this.level > level) return null; const timestamp = this.includeTimestamp ? new Date().toISOString() : ''; const message = args.map(arg => typeof arg === 'object' ? JSON.stringify(arg, null, 2) : String(arg) ).join(' '); const colored = this.enableColors ? this.colorize(levelName, message) : message; const prefix = `${this.prefix} [${levelName}]`; return timestamp ? `${timestamp} ${prefix} ${colored}` : `${prefix} ${colored}`; } /** * 终端 colors */ colorize(level, message) { const colors = { DEBUG: '\x1b[36m', INFO: '\x1b[32m', WARN: '\x1b[33m', ERROR: '\x1b[31m', }; const reset = '\x1b[0m'; return `${colors[level] || ''}${message}${reset}`; } /** * Write to log file */ writeToFile(formatted) { if (!this.logFile || !formatted) return; const plain = formatted.replace(/\x1b\[[0-9;]*m/g, ''); appendFileSync(this.logFile, plain + '\n'); } ``` ### Technical Analysis The logger serializes arbitrary object arguments and writes them verbatim to a caller-selected plaintext file. It removes ANSI color sequences but does not redact sensitive fields or sanitize embedded newline and control characters. A token, password, authorization header, cookie, private key, or sensitive task context passed as a logged object can therefore be retained in plaintext. Attacker-controlled newline characters can create forged multiline entries that obscure the actual event source or severity. The logger also lacks explicit owner-only file permissions, path restrictions, log rotation, and file-size limits. ### Attack Path 1. Sensitive or attacker-controlled data reaches `debug()`, `info()`, `warn()`, `error()`, or `logStructured()`. 2. Object values are serialized ...[truncated 814 chars]
Remediation
<![CDATA[ ## Remediation Suggestions - Apply recursive redaction to common secret fields, including passwords, tokens, API keys, cookies, authorization headers, and private-key material. - Replace embedded newline, carriage-return, escape, and other unsafe control characters in untrusted strings. - Prefer structured JSON logs with one validated event per line. - Create log files with mode `0600` and parent directories with mode `0700`. - Resolve and constrain log destinations to an approved logging directory. - Reject symlink destinations where feasible. - Add per-entry size limits, rotation, retention limits, and disk-usage monitoring. - Avoid logging complete task contexts, environment objects, request headers, or persisted session state by default. ]]>
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
Findings (59)

Ae1

High
Category
analysis-evasion
Content
node tests/test-suite.js
Confidence
100% confidence
Finding
Referenced artifact was not completely inspected

Ae1

High
Category
analysis-evasion
Content
node examples/harness-demo.js
Confidence
100% confidence
Finding
Referenced artifact was not completely inspected

Ae1

High
Category
analysis-evasion
Content
- [WORKFLOWS.md](./WORKFLOWS.md) - 工作流使用指南
Confidence
100% confidence
Finding
Referenced artifact was not completely inspected

Ae1

High
Category
analysis-evasion
Content
- [SELF-IMPROVEMENT.md](./SELF-IMPROVEMENT.md) - 自进化系统
Confidence
100% confidence
Finding
Referenced artifact was not completely inspected

Natural-Language Policy Violations

Medium
Confidence
93% confidence
Finding
This markdown file is written entirely in Chinese and presents the skill documentation as the required interaction language, without indicating that other languages are supported or that Chinese is a justified locale constraint. Under the policy, forcing a specific language without user opt-in is a natural-language policy violation.

Natural-Language Policy Violations

Medium
Confidence
87% confidence
Finding
The file presents all user-facing guidance in Chinese, including headings, instructions, and trigger descriptions, with no indication that users can choose another language or that the skill is intentionally limited to a Chinese-speaking audience. This creates a natural-language locale constraint that is not opt-in or justified in the document.

Vague Triggers

Medium
Confidence
96% confidence
Finding
The trigger list is broad and contains generic terms such as "工作流", "代码审查", and "测试生成" that are likely to match many unrelated user requests. In a skill-routing context, this can cause unintended activation, overbroad interception of prompts, and incorrect delegation to a skill with orchestration and self-improvement capabilities, increasing the chance of unsafe or confusing behavior.

Natural-Language Policy Violations

Medium
Confidence
95% confidence
Finding
The prominent tagline is written in Chinese, and the rest of the README is also presented in Chinese without any indication that users may select another language or that the skill is intentionally limited to a Chinese-speaking context. This can violate language/locale policy when a skill imposes a specific language without user opt-in.

Intent-Code Divergence

Medium
Confidence
91% confidence
Finding
The statement that 'all user input is validated' is presented as a universal assurance without supporting evidence across the stated audit scope. Overstated validation claims in a security report can create false confidence, causing real input-handling weaknesses elsewhere in the system to be overlooked or accepted without verification.

Sudo/Root Execution

Medium
Category
Privilege Escalation
Content
1. **设置文件权限**
   ```bash
   chmod 600 memory/wal/*.log
   chmod 600 memory/buffer/*.json
   ```
Confidence
80% confidence
Finding
Commands invoke sudo or root privileges. Verify this elevated access is necessary and justified.

Sudo/Root Execution

Medium
Category
Privilege Escalation
Content
1. **设置文件权限**
   ```bash
   chmod 600 memory/wal/*.log
   chmod 600 memory/buffer/*.json
   ```
Confidence
80% confidence
Finding
Commands invoke sudo or root privileges. Verify this elevated access is necessary and justified.

Intent-Code Divergence

Medium
Confidence
96% confidence
Finding
The document makes contradictory security claims: it marks file-permission control as compliant in the OWASP table while elsewhere stating access control is missing and recommending chmod 600. In a security audit, inconsistent assertions can mislead operators into believing a control is already implemented, which may delay remediation of real exposure around readable WAL or buffer files.

Missing User Warnings

Medium
Confidence
92% confidence
Finding
The document promotes persistent WAL logs, buffers, and learning/error files that store task state, progress, file names, and error details, but it does not warn that these artifacts may retain sensitive project context or secrets on disk for extended periods. In an agent skill, this increases the chance of unintentional data exposure through local files, backups, or later reuse by other tasks.

Natural-Language Policy Violations

Medium
Confidence
91% confidence
Finding
The natural-language content of the skill description is entirely in Chinese, including the title tagline and usage guidance, with no indication that another language can be selected. Under the policy, forcing a specific language without user opt-in is a locale/language policy violation unless clearly justified.

Missing User Warnings

Medium
Confidence
90% confidence
Finding
The skill includes a CI/CD publishing example with system-affecting actions such as build, test, and deploy steps, but it does not explicitly warn the user that these actions can modify environments or trigger production changes. In an agent skill context, examples strongly influence downstream automation behavior, so omission of safety guidance can lead to accidental execution of deployment-like workflows without adequate confirmation or sandboxing.

Intent-Code Divergence

Medium
Confidence
90% confidence
Finding
Lines L019-L025 describe the workflows as reusable production-grade processes with integrated professional agents and complete error handling. However, L486 explicitly lists "integrate real Agent implementation" as a future step, which contradicts the claim that those capabilities are already present.

Intent-Code Divergence

Medium
Confidence
79% confidence
Finding
The document lists specific available workflow files in L027-L035 and again provides a concrete file structure with sizes in L469-L480, implying implemented artifacts. But L486 says real agent integration is still pending, creating a contradiction between the documentation's claimed readiness and its stated future work.

Missing User Warnings

Medium
Confidence
93% confidence
Finding
该 markdown 文件适用 SQP-2。L124 明确声明“自动保存测试文件”,L143-L145 又展示了直接调用 saveTests 将内容写入 `__tests__` 目录,但文档没有提醒这会修改工作区文件、可能覆盖现有测试,或建议先审查生成内容。对于会落盘写文件的行为,缺少这类用户可见警示属于文档层面的安全提示缺失。

Natural-Language Policy Violations

Medium
Confidence
90% confidence
Finding
L200 和 L241 将文档生成语言固定为 `zh-CN`,而文档中未说明这是可选默认值、未给出用户语言选择,也未表明该技能仅限中文场景。根据 SQP-3,强制特定语言/locale 且无用户 opt-in 属于自然语言政策违规。

Missing User Warnings

Medium
Confidence
92% confidence
Finding
This section promotes automatic generation and saving of CI/CD and deployment artifacts, including deploy scripts and production-targeted configuration, without emphasizing review gates or integrity risks. In an agent skill context, this is more dangerous because generated pipeline files can alter build, deployment, or secret-handling behavior and may be trusted by users as ready for production use.

Autonomous Decision Making

Medium
Category
Excessive Agency
Content
const workflow = new CICDWorkflow({
  platform: 'github',     // github | gitlab | jenkins
  target: 'production',   // staging | production
  autoDeploy: true,
  enableNotifications: true,
});
Confidence
87% confidence
Finding
The example enables 'autoDeploy: true' while targeting production, normalizing autonomous deployment behavior without an explicit approval checkpoint. In a workflow skill, this is security-relevant because an agent-generated or agent-triggered production deployment can propagate faulty or malicious changes directly into live systems.

Natural-Language Policy Violations

Medium
Confidence
87% confidence
Finding
The natural-language description is written entirely in Chinese, while the rest of the manifest uses English metadata. This creates an implicit language constraint for users without offering a language choice or documenting that the skill is intended only for a Chinese-speaking or region-specific audience.

Natural-Language Policy Violations

Medium
Confidence
94% confidence
Finding
The file’s user-facing description and execution instructions are written in Chinese, and the rest of the demo output is also hard-coded in Chinese. This imposes a specific language/locale on users without any visible opt-in or alternative, which matches the language/locale policy violation criteria.

Missing User Warnings

Medium
Confidence
87% confidence
Finding
The sequential release example includes a final step to push to production, which is a safety-critical operation. In markdown guidance, such behavior should be accompanied by a user-facing warning about production impact, but no such warning appears in this section.

Missing User Warnings

Medium
Confidence
89% confidence
Finding
This markdown file includes an example CI/CD pipeline with tasks that deploy to staging and production, which are safety-critical operations that can affect live systems. The surrounding documentation presents these actions as routine examples but does not include any warning, confirmation note, or caution about their impact.

Static analysis

No suspicious patterns detected.