Dreaming Extractor

Daily fact extraction from AI agent session history into a persistent learned.md memory file

Security Scan

The skill's code, instructions, and requirements are consistent with its stated purpose (collect local session logs, let the agent extract facts, and append them to a local learned.md); nothing requests unrelated credentials or external downloads, but it will read potentially sensitive local session files and cause LLM calls via the agent's model runtime.

Purpose & Capability

Name/description (daily extraction from agent session history into learned.md) matches what the files and SKILL.md do. Required binary: python3 is appropriate for the included Python scripts. No unrelated credentials, binaries, or surprising config paths are requested.

Instruction Scope

Instructions are narrowly scoped to: collect yesterday's session JSONL files, run a local budget check, have the agent extract facts from the collected text, append results to memory/learned.md, and log budget. One important note: there is no included script that calls an external LLM — the SKILL.md expects the agent (LLM) itself to perform extraction using the collected input. That means the collected session contents will be processed by whatever model/runtime the agent uses (and thus may be sent to the configured model provider). The scripts themselves only read/write local files and do not make network calls.

Install Mechanism

Instruction-only skill with two small local Python scripts; no install spec, no downloads, and nothing written to unexpected system locations beyond the user's OpenClaw workspace and agent session directories. Lowest-risk install profile.

Credentials

No required environment variables or credentials declared. The scripts optionally respect OPENCLAW_WORKSPACE and OPENCLAW_AGENTS_DIR if present; that is reasonable and proportional. No unrelated secrets are requested.

Persistence & Privilege

always:false (not forced-in) and disable-model-invocation:false (normal — agent may call it). The skill writes to its own workspace memory/learned.md, which is expected. It does not modify other skills or system-wide settings.

Assessment

This skill appears to do what it says: it reads your local OpenClaw agent session JSONL files, prepares a cleaned text input, and expects the agent/model to extract facts which are appended to memory/learned.md. Before installing, consider: 1) session files often contain sensitive content (credentials, private messages). Installing this skill will cause those contents to be processed by your agent/model runtime — ensure you trust the configured model provider and understand where model data is sent. 2) The provided scripts do not themselves call any external API; the factual extraction step is performed by the agent/model (so costs and data flow depend on your model configuration). 3) Review and restrict file permissions on ~/.openclaw/agents and your workspace if you do not want extracted facts persisted. 4) The budget.py enforces a $2/day hard cap in local bookkeeping, but real costs depend on your model provider billing — verify your model settings. If you are comfortable with local file reads and the agent using your configured model, the skill is coherent and reasonable to use.

Skill: Dreaming Extractor

Reads yesterday's real agent conversations, extracts structured facts using an LLM, and appends them to memory/learned.md. Designed to run daily via cron — builds a persistent, searchable knowledge base from your agent's actual interactions.

What it does

• Scans yesterday's session JSONL files from ~/.openclaw/agents/
• Filters out system noise (startup sequences, dreaming sessions, tool output)
• Extracts structured facts: decisions made, problems solved, config changes, corrections
• Appends extracted facts to memory/learned.md with confidence scores and source citations
• Enforces a daily token budget to cap cost

Execution Steps

When triggered (by cron or manually), follow these steps:

Step 1: Collect sessions

python3 skills/skill-dreaming-extractor/scripts/collect.py

Outputs a clean text file at memory/.dreams/extraction-input/YYYY-MM-DD.txt containing yesterday's real conversations. If output is "No real sessions found", stop here.

Step 2: Check budget

python3 skills/skill-dreaming-extractor/scripts/budget.py --check

If output is "BUDGET EXCEEDED", stop. Do not proceed.

Step 3: Extract facts

Read the collected input. For each meaningful exchange, extract structured facts:

{
  "facts": [
    {
      "subject": "specific entity or concept (e.g., 'API endpoint', 'deployment target')",
      "predicate": "what happened or was decided (e.g., 'was deployed to', 'was updated to')",
      "object": "the outcome or target (e.g., 'production on 2026-04-14', 'v2 with retry logic')",
      "date": "YYYY-MM-DD",
      "confidence": 0.0-1.0,
      "source": "session-id:line-range"
    }
  ]
}

Extraction Rules

INCLUDE:

• Decisions made (architectural, business, operational)
• Problems solved and how
• Configuration changes with rationale
• New capabilities deployed or verified
• Corrections or feedback from the user
• External facts learned (API changes, service incidents, pricing)

REJECT (scaffolding — never extract):

• Session startup greetings / "back online" messages
• Tool call results (file contents, grep output, git status)
• Progress updates ("working on X", "let me check")
• Repetitions of the same fact across sessions
• Vague themes with no concrete data
• Meta-conversation about the conversation itself
• Token counts, cost reports, cache stats

Quality gates:

• Each fact MUST have at least 3 concrete tokens (subject + predicate + date minimum)
• Confidence 0.9+ = directly stated by user or confirmed by system output
• Confidence 0.7-0.89 = strongly implied, single source
• Confidence 0.5-0.69 = inferred, use sparingly
• Below 0.5 = do not extract

Step 4: Write to learned.md

Append extracted facts to memory/learned.md:

## Learned — YYYY-MM-DD

- **[subject]** [predicate] [object] | confidence: X.XX | source: [session-id:lines]
- ...

If the file doesn't exist, create it with a header first.

Step 5: Log budget

python3 skills/skill-dreaming-extractor/scripts/budget.py --log --facts-count N

Where N = number of facts extracted.

Step 6: Report

Output a summary:

Dreaming extraction complete for YYYY-MM-DD:
- Sessions processed: X
- Facts extracted: Y
- Confidence range: X.XX - X.XX
- Budget remaining: $X.XX / $2.00

Manual Run

python3 skills/skill-dreaming-extractor/scripts/collect.py --date 2026-04-14
# Then follow steps 2-6

Cost

• Estimated: ~$0.50–1.50/day (Sonnet, ~30–200K chars of session history)
• Hard cap: $2.00/day (configurable in budget.py)
• Monthly projection: ~$15–45

Cron Setup

openclaw cron add "Dreaming Extractor" "3 3 * * *" "Run task spec: skills/skill-dreaming-extractor/SKILL.md"

Fires daily at 03:03 UTC (after sessions have closed for the day).

Success Metrics

Metric	Target
Facts per cycle with 3+ concrete tokens	≥ 3
Confidence variance (std dev)	≥ 0.15
Scaffolding ratio	< 10%
Citations per week (facts actually recalled)	≥ 1

If citation count stays at 0 for 30 days, the system isn't adding value — retire it.