#learning #hermes #self-improving #self-improvement

Hermes Learning Loop

Self-improving learning loop inspired by Hermes Agent. Automatically extracts successful workflows, creates skills, and persists knowledge across sessions.

陈子昂@mystour

Install

openclaw skills install @mystour/hermes-learning-loop

Hermes Learning Loop for OpenClaw

Inspired by: NousResearch/hermes-agent (Self-improving AI agent)

Implements Hermes Agent's core learning loop in OpenClaw — automatically extracts successful workflows, creates reusable skills, and persists curated knowledge across sessions.

When to Use

After completing complex multi-step tasks
When you discover a workflow worth repeating
Automatically via heartbeat (periodic nudge)
When user says "remember this" or "save this workflow"

Features

Periodic Nudge — Auto-reflect every N tasks (default: 5)
Skill Extraction — Convert successful workflows to reusable skills
4-Layer Memory — Prompt memory + Session search + Skills + User modeling
Curated Memory — Agent decides what's worth keeping (not logging everything)
Progressive Disclosure — Only load skill summaries by default, full content on-demand
FTS5 Session Search — SQLite-powered historical context retrieval

Quick Start

Install

bash

clawhub install hermes-learning-loop

Manual Trigger

bash

# After completing a task
node learning-loop.js extract --session=<session_id>

# Create skill from workflow
node learning-loop.js create-skill --name="my-skill" --description="What it does"

# Periodic nudge (heartbeat)
node learning-loop.js nudge

Auto-Trigger (Integration)

Add to HEARTBEAT.md:

markdown

## Learning Loop - Periodic Nudge

**Frequency:** Every 5 tasks or 30 minutes

**Task:**
1. Run `node learning-loop.js nudge`
2. Review extracted memories
3. Approve/reject skill creations

Architecture

Learning Loop Cycle

text

┌─────────────────────────────────────────────────────────┐
│  1. Task Execution                                      │
│     - Agent completes task                              │
│     - Track tool calls, decisions, outcomes             │
└──────────────┬──────────────────────────────────────────┘
               │
               ▼
┌─────────────────────────────────────────────────────────┐
│  2. Periodic Nudge (every 5 tasks)                      │
│     - System prompt: "Reflect on recent activity"       │
│     - Agent evaluates: Is this worth persisting?        │
└──────────────┬──────────────────────────────────────────┘
               │
               ▼
┌─────────────────────────────────────────────────────────┐
│  3. Memory Extraction                                   │
│     - If valuable → Write to MEMORY.md / USER.md        │
│     - If workflow → Create skill file                   │
│     - If context → Index in session archive             │
└──────────────┬──────────────────────────────────────────┘
               │
               ▼
┌─────────────────────────────────────────────────────────┐
│  4. Skill Creation                                      │
│     - Extract steps, tool calls, file references        │
│     - Write to ~/.openclaw/skills/<category>/<name>/   │
│     - Follow agentskills.io specification               │
└─────────────────────────────────────────────────────────┘

4-Layer Memory System

Layer	Purpose	Location	Load Timing
Prompt Memory	Always-on context	`MEMORY.md`, `USER.md`	Every session start
Skills	Procedural memory (how-to)	`~/.openclaw/skills/`	On-demand (summary → full)
Session Archive	Episodic memory (what happened)	SQLite + FTS5	Deliberate retrieval
User Model	Behavioral patterns	Optional (Honcho-like)	Continuous passive

Skill Triggers

Automatically create skill when:

✅ 5+ tool calls in sequence
✅ Recovered from error successfully
✅ User corrected → fixed approach
✅ Non-obvious workflow that worked
✅ Repeated pattern detected (3+ times)

Usage Examples

Example 1: Post-Task Skill Extraction

bash

# After completing complex deployment
export SESSION_ID="2026-04-03-deploy"
export TASK_OUTCOME="success"
export TOOL_CALLS="15"

node learning-loop.js extract

Output:

text

📊 Learning Loop Analysis

✅ Task completed successfully
📈 Tool calls: 15
⚠️  Error recovery: 2 instances
💡  User corrections: 1

🎯 Skill-worthy detected: YES

Proposed skill:
  Name: deploy-to-k8s
  Description: Deploy application to Kubernetes with health checks
  Steps: 7
  Tool calls: 15

📄 Created: ~/.openclaw/skills/devops/deploy-to-k8s/SKILL.md

Example 2: Periodic Nudge

bash

# Heartbeat triggers this
node learning-loop.js nudge

Output:

text

🔔 Periodic Nudge - Task #5

Looking back at tasks 1-5...

Task 1: ✅ Simple (no extraction needed)
Task 2: ✅ Complex workflow detected
  → Extracted: "github-pr-workflow"
Task 3: ✅ Error recovery pattern
  → Extracted: "debug-python-import"
Task 4: ✅ Simple
Task 5: ✅ User correction applied
  → Updated: "github-pr-workflow" (v1.0.1)

📊 Summary:
  - New skills: 2
  - Updated skills: 1
  - Memory entries: 3

Example 3: Memory Curation

bash

# Review what's worth keeping
node learning-loop.js curate --session="2026-04-03"

Decision Framework:

text

┌──────────────────────────────────────────────────────┐
│ Should this be persisted?                            │
├──────────────────────────────────────────────────────┤
│ ❌ Chat pleasantries → Discard                       │
│ ❌ Failed attempts → Discard (keep only success)     │
│ ✅ Successful workflows → Extract as skill           │
│ ✅ User preferences → Add to USER.md                 │
│ ✅ Project context → Add to MEMORY.md (project/)     │
│ ✅ Corrections/feedback → Add to MEMORY.md (feedback/)│
└──────────────────────────────────────────────────────┘

Configuration

Environment Variables

Variable	Description	Default
`LEARNING_NUDGE_INTERVAL`	Tasks between nudges	`5`
`LEARNING_MIN_TOOL_CALLS`	Min tool calls for skill	`5`
`LEARNING_AUTO_CREATE`	Auto-create skills (vs approve)	`false`
`LEARNING_SKILLS_DIR`	Skills directory	`~/.openclaw/skills/`
`LEARNING_MEMORY_DIR`	Memory directory	`~/.openclaw/memory/`

Config File

yaml

# ~/.openclaw/learning-loop.yaml
learning:
  nudge:
    enabled: true
    interval: 5  # tasks
    
  skill_creation:
    auto_approve: false  # Review before creating
    min_tool_calls: 5
    categories:
      - devops
      - research
      - productivity
      
  memory:
    max_prompt_chars: 3575  # Hermes-style tight limit
    archive_enabled: true
    fts5_enabled: true

Skill File Format

Follows agentskills.io specification:

markdown

---
name: deploy-to-k8s
description: Deploy application to Kubernetes with health checks
version: 1.0.0
platforms: [linux, macos]
metadata:
  tags: [kubernetes, devops, deployment]
  category: devops
  created_from: session-2026-04-03
---

# deploy-to-k8s

## Overview

Deploy any application to Kubernetes cluster with automated health checks and rollback.

## Prerequisites

- kubectl configured
- Kubernetes cluster access
- Docker image ready

## Steps

1. **Validate cluster connection**
   ```bash
   kubectl cluster-info

Apply deployment manifest
bash
```
kubectl apply -f deployment.yaml
```
Wait for rollout
bash
```
kubectl rollout status deployment/app
```
Health check
bash
```
kubectl get pods -l app=myapp
```
Verify service
bash
```
kubectl get svc app-service
```

Tool Calls Used

exec: kubectl commands
read: deployment.yaml
web_search: Kubernetes docs (if errors)

Related Skills

docker-build-optimization
k8s-troubleshooting

text


## Integration with OpenClaw

### Heartbeat Integration

```markdown
## HEARTBEAT.md Update

### Learning Loop - Periodic Nudge

**Frequency:** Every 5 tasks

**Steps:**
1. Check task counter
2. If counter % 5 == 0 → Run nudge
3. Review proposed skills/memories
4. Approve/reject
5. Reset counter

PUA Integration

bash

# Task PUA can trigger learning loop
if task.failed && task.recovered:
  node learning-loop.js extract --reason="error-recovery"
  
if task.pua_level >= "L2":
  # Complex task → likely skill-worthy
  node learning-loop.js nudge

Memory PUA Integration

bash

# Memory health check includes learning loop status
node memory-pua.js audit

# Output includes:
# - Skills created this week
# - Memories curated
# - Nudge effectiveness

Comparison: Hermes vs This Implementation

Feature	Hermes Agent	This Skill
Periodic Nudge	✅ Built-in	✅ Via heartbeat
Skill Auto-Creation	✅ Full auto	⚠️ Opt-in approval
4-Layer Memory	✅ SQLite + FTS5	⚠️ Markdown + optional SQLite
Progressive Disclosure	✅ Summary → Full	✅ Same pattern
User Modeling (Honcho)	✅ Optional	❌ Not implemented
Gateway Integration	✅ Multi-platform	⚠️ OpenClaw channels only
Context Compression	✅ Lineage-aware	⚠️ Basic summarization

Benefits

✅ Curated not dumped — Agent decides what's worth keeping
✅ Token-efficient — Progressive disclosure keeps context small
✅ Portable skills — Follows agentskills.io standard
✅ Self-improving — Gets better the more you use it
✅ OpenClaw native — Works with existing memory system

Related Skills

claude-memory-optimizer: Memory structure and migration
task-pua: Task persistence and quality
memory-pua: Memory health maintenance

References

Hermes Agent: https://github.com/NousResearch/hermes-agent
Hermes Docs: https://hermes-agent.nousresearch.com/
agentskills.io: https://agentskills.io/specification
FTS5 Docs: https://www.sqlite.org/fts5.html

License

MIT-0

Version 1.0.0: Initial implementation inspired by Hermes Agent learning loop

Hermes Learning Loop

Install

Hermes Learning Loop for OpenClaw

When to Use

Features

Quick Start

Install

Manual Trigger

Auto-Trigger (Integration)

Architecture

Learning Loop Cycle

4-Layer Memory System

Skill Triggers

Usage Examples

Example 1: Post-Task Skill Extraction

Example 2: Periodic Nudge

Example 3: Memory Curation

Configuration

Environment Variables

Config File

Skill File Format

Tool Calls Used

Related Skills

PUA Integration

Memory PUA Integration

Comparison: Hermes vs This Implementation

Benefits

Related Skills

References

License

Related skills