Back to skill

Security audit

Dobby Harness Self-improving Coding Skills

Security checks for vulnerabilities and agentic risk

Overview

This skill is mostly coherent, but it persists broad agent/session data to local plaintext files and gives broad automation guidance that should be reviewed before use.

Review this before installing in sensitive workspaces. Avoid storing secrets, tokens, full environment dumps, private code context, or personal data in the WAL, buffer, or logs unless you add restrictive file permissions, redaction, retention limits, and encryption. Treat CI/CD and PR-commenting examples as requiring explicit user approval and safe target environments.

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (4)

T09 · Insecure Skill Coding Practices

Warning
Location
memory/wal.js:78
Finding

WAL checksum verification always accepts modified entries

Content
View full analysis
Remediation
View remediation

T09 · Insecure Skill Coding Practices

Warning
Location
memory/wal.js:321
Finding

WAL persists unrestricted task data in plaintext files

Content
View full analysis
Remediation
View remediation

T09 · Insecure Skill Coding Practices

Warning
Location
memory/working-buffer.js:244
Finding

Working Buffer stores arbitrary session state in an unhardened plaintext JSON file

Content
View full analysis
Remediation
View remediation

T09 · Insecure Skill Coding Practices

Note
Location
harness/utils/logger.js:47
Finding

File logger writes unrestricted objects without secret redaction or secure file controls

Content
View full analysis
level) return null; const timestamp = this.includeTimestamp ? new Date().toISOString() : ''; const message = args.map(arg => typeof arg === 'object' ? JSON.stringify(arg, null, 2) : String(arg) ).join(' '); const colored = this.enableColors ? this.colorize(levelName, message) : message; const prefix = `${this.prefix} [${levelName}]`; return timestamp ? `${timestamp} ${prefix} ${colored}` : `${prefix} ${colored}`; } /** * 终端 colors */ colorize(level, message) { const colors = { DEBUG: '\x1b[36m', INFO: '\x1b[32m', WARN: '\x1b[33m', ERROR: '\x1b[31m', }; const reset = '\x1b[0m'; return `${colors[level] || ''}${message}${reset}`; } /** * Write to log file */ writeToFile(formatted) { if (!this.logFile || !formatted) return; const plain = formatted.replace(/\x1b\[[0-9;]*m/g, ''); appendFileSync(this.logFile, plain + '\n'); } ``` ### Technical Analysis The logger serializes arbitrary object arguments and writes them verbatim to a caller-selected plaintext file. It removes ANSI color sequences but does not redact sensitive fields or sanitize embedded newline and control characters. A token, password, authorization header, cookie, private key, or sensitive task context passed as a logged object can therefore be retained in plaintext. Attacker-controlled newline characters can create forged multiline entries that obscure the actual event source or severity. The logger also lacks explicit owner-only file permissions, path restrictions, log rotation, and file-size limits. ### Attack Path 1. Sensitive or attacker-controlled data reaches `debug()`, `info()`, `warn()`, `error()`, or `logStructured()`. 2. Object values are serialized ...[truncated 814 chars]
Remediation
View remediation
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
Findings (59)

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 232)May include surrounding context.

md
node tests/test-suite.js

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 238)May include surrounding context.

md
node examples/harness-demo.js

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 244)May include surrounding context.

md
- [WORKFLOWS.md](./WORKFLOWS.md) - 工作流使用指南

Ae1

High
Category
analysis-evasion
Confidence
100% confidence
Finding

Referenced artifact was not completely inspected

Content

Scanner excerpt · SKILL.md (reported line 245)May include surrounding context.

md
- [SELF-IMPROVEMENT.md](./SELF-IMPROVEMENT.md) - 自进化系统

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

This markdown file is written entirely in Chinese and presents the skill documentation as the required interaction language, without indicating that other languages are supported or that Chinese is a justified locale constraint. Under the policy, forcing a specific language without user opt-in is a natural-language policy violation.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
87% confidence
Finding

The file presents all user-facing guidance in Chinese, including headings, instructions, and trigger descriptions, with no indication that users can choose another language or that the skill is intentionally limited to a Chinese-speaking audience. This creates a natural-language locale constraint that is not opt-in or justified in the document.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
96% confidence
Finding

The trigger list is broad and contains generic terms such as "工作流", "代码审查", and "测试生成" that are likely to match many unrelated user requests. In a skill-routing context, this can cause unintended activation, overbroad interception of prompts, and incorrect delegation to a skill with orchestration and self-improvement capabilities, increasing the chance of unsafe or confusing behavior.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

The prominent tagline is written in Chinese, and the rest of the README is also presented in Chinese without any indication that users may select another language or that the skill is intentionally limited to a Chinese-speaking context. This can violate language/locale policy when a skill imposes a specific language without user opt-in.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

The statement that 'all user input is validated' is presented as a universal assurance without supporting evidence across the stated audit scope. Overstated validation claims in a security report can create false confidence, causing real input-handling weaknesses elsewhere in the system to be overlooked or accepted without verification.

Content

No source excerpt is available for this finding.

Sudo/Root Execution

Medium
Category
Privilege Escalation
Confidence
80% confidence
Finding

Commands invoke sudo or root privileges. Verify this elevated access is necessary and justified.

Content

Scanner excerpt · SECURITY-AUDIT.md (reported line 270)May include surrounding context.

  1. 设置文件权限
    bash
    chmod 600 memory/wal/*.log
    chmod 600 memory/buffer/*.json
    

Sudo/Root Execution

Medium
Category
Privilege Escalation
Confidence
80% confidence
Finding

Commands invoke sudo or root privileges. Verify this elevated access is necessary and justified.

Content

Scanner excerpt · SECURITY-AUDIT.md (reported line 271)May include surrounding context.

  1. 设置文件权限
    bash
    chmod 600 memory/wal/*.log
    chmod 600 memory/buffer/*.json
    

Intent-Code Divergence

Medium
Category
Not specified by scanner
Confidence
96% confidence
Finding

The document makes contradictory security claims: it marks file-permission control as compliant in the OWASP table while elsewhere stating access control is missing and recommending chmod 600. In a security audit, inconsistent assertions can mislead operators into believing a control is already implemented, which may delay remediation of real exposure around readable WAL or buffer files.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
92% confidence
Finding

The document promotes persistent WAL logs, buffers, and learning/error files that store task state, progress, file names, and error details, but it does not warn that these artifacts may retain sensitive project context or secrets on disk for extended periods. In an agent skill, this increases the chance of unintentional data exposure through local files, backups, or later reuse by other tasks.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

The natural-language content of the skill description is entirely in Chinese, including the title tagline and usage guidance, with no indication that another language can be selected. Under the policy, forcing a specific language without user opt-in is a locale/language policy violation unless clearly justified.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
90% confidence
Finding

The skill includes a CI/CD publishing example with system-affecting actions such as build, test, and deploy steps, but it does not explicitly warn the user that these actions can modify environments or trigger production changes. In an agent skill context, examples strongly influence downstream automation behavior, so omission of safety guidance can lead to accidental execution of deployment-like workflows without adequate confirmation or sandboxing.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Medium
Category
Not specified by scanner
Confidence
90% confidence
Finding

Lines L019-L025 describe the workflows as reusable production-grade processes with integrated professional agents and complete error handling. However, L486 explicitly lists "integrate real Agent implementation" as a future step, which contradicts the claim that those capabilities are already present.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Medium
Category
Not specified by scanner
Confidence
79% confidence
Finding

The document lists specific available workflow files in L027-L035 and again provides a concrete file structure with sizes in L469-L480, implying implemented artifacts. But L486 says real agent integration is still pending, creating a contradiction between the documentation's claimed readiness and its stated future work.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

该 markdown 文件适用 SQP-2。L124 明确声明“自动保存测试文件”,L143-L145 又展示了直接调用 saveTests 将内容写入 __tests__ 目录,但文档没有提醒这会修改工作区文件、可能覆盖现有测试,或建议先审查生成内容。对于会落盘写文件的行为,缺少这类用户可见警示属于文档层面的安全提示缺失。

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
90% confidence
Finding

L200 和 L241 将文档生成语言固定为 zh-CN,而文档中未说明这是可选默认值、未给出用户语言选择,也未表明该技能仅限中文场景。根据 SQP-3,强制特定语言/locale 且无用户 opt-in 属于自然语言政策违规。

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
92% confidence
Finding

This section promotes automatic generation and saving of CI/CD and deployment artifacts, including deploy scripts and production-targeted configuration, without emphasizing review gates or integrity risks. In an agent skill context, this is more dangerous because generated pipeline files can alter build, deployment, or secret-handling behavior and may be trusted by users as ready for production use.

Content

No source excerpt is available for this finding.

Autonomous Decision Making

Medium
Category
Excessive Agency
Confidence
87% confidence
Finding

The example enables 'autoDeploy: true' while targeting production, normalizing autonomous deployment behavior without an explicit approval checkpoint. In a workflow skill, this is security-relevant because an agent-generated or agent-triggered production deployment can propagate faulty or malicious changes directly into live systems.

Content

Scanner excerpt · WORKFLOWS.md (reported line 265)May include surrounding context.

md
const workflow = new CICDWorkflow({
  platform: 'github',     // github | gitlab | jenkins
  target: 'production',   // staging | production
  autoDeploy: true,
  enableNotifications: true,
});

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
87% confidence
Finding

The natural-language description is written entirely in Chinese, while the rest of the manifest uses English metadata. This creates an implicit language constraint for users without offering a language choice or documenting that the skill is intended only for a Chinese-speaking or region-specific audience.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

The file’s user-facing description and execution instructions are written in Chinese, and the rest of the demo output is also hard-coded in Chinese. This imposes a specific language/locale on users without any visible opt-in or alternative, which matches the language/locale policy violation criteria.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
87% confidence
Finding

The sequential release example includes a final step to push to production, which is a safety-critical operation. In markdown guidance, such behavior should be accompanied by a user-facing warning about production impact, but no such warning appears in this section.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
89% confidence
Finding

This markdown file includes an example CI/CD pipeline with tasks that deploy to staging and production, which are safety-critical operations that can affect live systems. The surrounding documentation presents these actions as routine examples but does not include any warning, confirmation note, or caution about their impact.

Content

No source excerpt is available for this finding.