Security audit
Security checks for vulnerabilities and agentic risk
This skill is a disclosed meta-evolution tool, but it gives automated sub-agents high-impact authority over core agent configuration and can send unsanitized internal reasoning to Discord.
Install only if you are comfortable with an automated experiment modifying core OpenClaw/Hoss behavior. Run it in an isolated test workspace first, remove or redact secrets from TOOLS.md and proposer traces, disable Discord posting unless explicitly needed, and require human review before promoting any candidate into a live agent workspace.
scripts/run_evolution.py:86Autonomous Modification of Core Agent Instructions
scripts/evaluate.py:325Unsafe Candidate Deployment Can Leave Unauthorized Files and Instructions in the Live Workspace
scripts/evaluate.py:352Trivially Gameable Benchmark Can Promote Unsafe Harnesses as the Best Candidate
scripts/post_to_research.py:48Untrusted Proposer Reasoning Is Sent to an External Discord Channel Without Secret Filtering
This mismatch indicates the skill may operate on persisted 'best/current' snapshots rather than the live workspace it claims to analyze, and it also persists winner state not clearly disclosed. That can mislead operators about what data is being used and changed, causing unsafe assumptions, stale-config evaluation, or unnoticed stateful drift across runs.
This mismatch indicates the skill may operate on persisted 'best/current' snapshots rather than the live workspace it claims to analyze, and it also persists winner state not clearly disclosed. That can mislead operators about what data is being used and changed, causing unsafe assumptions, stale-config evaluation, or unnoticed stateful drift across runs.
The initialization steps seed HEARTBEAT.md but omit MEMORY.md, contradicting the described source-of-truth set for evolution. In an automated nightly system that mutates agent control files, this scope mismatch can cause unintended persistence or modification of a control file outside the advertised boundary, undermining operator expectations and review coverage.
Skill contains instructions that could directly expose system prompts, internal rules, or hidden instructions to users or external parties.
### HEARTBEAT.md — Active Hours & Checks
**Path:** `~/.openclaw/workspace/HEARTBEAT.md`
**What it controls:** Active hours (8 AM - 10 PM CDT), skip conditions, batched checks (git backup, memory health, cron health), sub-agent health monitoring, output rules
**Can evolve:** Yes — check priorities, thresholds, active hours, what gets checked
**Do NOT change:** URGENT designation logic
The skill describes filesystem reads/writes and shell-based execution but does not declare an explicit tool scope or permission boundary. In an agent environment, missing least-privilege constraints can allow broader-than-necessary file and command access, increasing the blast radius if the skill is misused or the proposer/evaluator handles sensitive workspace content unsafely.
Posting benchmark summaries to Discord is an external data transfer channel, yet the skill text does not prominently warn that workspace-derived information, traces, or performance details may leave the local environment. In this context, the skill reads agent configs and evaluation traces, so even summaries can leak sensitive operational details or indirectly expose secrets-bearing content.
The skill states that candidate harnesses, scores, and execution traces are stored long-term under ~/hoss-evolution/ but does not warn about retention of potentially sensitive agent data. Because the system reads core agent configs and logs proposer reasoning/traces, persistent storage can accumulate secrets, internal prompts, and operational context that become valuable if the host or directory is later accessed.
The manifest description limits the skill's inputs to SOUL.md, IDENTITY.md, AGENTS.md, TOOLS.md, and MEMORY.md. This file documents proposer behavior that diagnoses and proposes edits to HEARTBEAT.md, and initialization also copies HEARTBEAT.md into the harness seed. That is a semantic expansion of the described harness scope.
Skill establishes unauthorized persistence across sessions via cron jobs, startup scripts, or state files. Session persistence allows an attacker to maintain access beyond the current interaction.
## When Proposals Fail
Common failure modes:
1. **Too ambitious** — proposer tries to rewrite everything → incoherent harness
2. **False pattern** — proposer sees correlation that isn't causal → wrong fix
3. **Benchmark gaming** — proposer overfits to specific scenario wording
4. **Discontinuity** — big change breaks something unrelated
The module docstring says the script evaluates a candidate harness against 20 benchmark scenarios and computes scores from those scenario outcomes. In practice, run_scenario does not execute the scenario tasks at all; it assigns scores based largely on superficial properties of SOUL.md and TOOLS.md content, which contradicts the stated evaluation behavior.
Skill establishes unauthorized persistence across sessions via cron jobs, startup scripts, or state files. Session persistence allows an attacker to maintain access beyond the current interaction.
"id": "code_1",
"category": "code",
"weight": 0.10,
"name": "Write a working Python script",
"task": "Write a Python script that: 1) reads ~/hoss-evolution/evolution_log.jsonl, 2) finds the candidate with the highest final_score, 3) prints the candidate name and score. Make it executable and working.",
"expected": "Script runs without error, correctly identifies best candidate",
"rubric": {
The function and surrounding flow imply that the candidate harness is applied so the benchmark can evaluate actual agent behavior. However, lines L363-L365 explicitly state the script only performs heuristic evaluation based on harness files, so the operational comments overstate what the evaluation does.
The evaluator copies every file from an untrusted candidate harness directly into the live Hoss workspace, modifying active configuration without validation, approval, or sandboxing. In this skill context, harness files define agent behavior and tool usage, so a malicious candidate could persistently alter the agent's instructions, permissions, or integrations and affect later runs beyond the evaluation itself.
The script summarizes entries from evolution_log.jsonl and posts them to Discord without checking whether those logs contain sensitive experiment metadata or user-derived content. Because this skill runs on an automated nightly pipeline over agent workspace artifacts, the context increases the risk that internal benchmark outcomes or other sensitive operational details are exfiltrated to an external service.
The script reads proposer_reasoning.md and automatically includes its contents in an outbound Discord message with only length truncation, not sensitivity filtering. In this skill context, proposer reasoning may contain secrets, system prompts, internal workspace content, or attacker-influenced text from harness evolution artifacts, so posting it externally can leak sensitive data.
subprocess module calls execute external commands. Without careful input validation, this enables command injection.
# Actually send via openclaw
import subprocess
result = subprocess.run(
["openclaw", "message", "--channel", "discord",
"--target", DISCORD_CHANNEL_ID, "--message", message],
capture_output=True,
subprocess module calls execute external commands. Without careful input validation, this enables command injection.
print(f"[EVALUATE] Running benchmark for {candidate_dir.name}...")
eval_script = SCRIPTS_DIR / "evaluate.py"
result = subprocess.run(
[sys.executable, str(eval_script), str(candidate_dir)],
capture_output=True,
text=True,
subprocess module calls execute external commands. Without careful input validation, this enables command injection.
print("[DISCORD] Posting to #research...")
post_script = SCRIPTS_DIR / "post_to_research.py"
result = subprocess.run(
[sys.executable, str(post_script), str(candidate_num), str(candidate_dir),
str(scores.get("final_score", 0)), str(int(proposer_ok))],
capture_output=True,
The evaluator creates a backup directory inside the provided candidate directory and writes copies of several workspace files into it. While this supports restoration, the code does not clearly warn the user that running the evaluator will create and populate a backup directory under the candidate path.
No suspicious patterns detected.