Tiered Memory

EvoClaw Tiered Memory Architecture v2.1.0 - LLM-powered three-tier memory system with structured metadata extraction, URL preservation, validation, and cloud...

Security Scan

The skill is generally what it claims to be (a three‑tier memory system), but there are inconsistencies and privacy/network risks you should understand before installing: it reads local notes, extracts URLs/commands/paths, and can upload sensitive hot memory to cloud storage and to arbitrary LLM endpoints — yet the registry metadata does not declare required credentials or endpoints.

Purpose & Capability

Name/description, README, SKILL.md and included scripts align: this is a three‑tier memory system with distillation, tree search, and optional Turso cold storage. Expectation of LLM endpoints and a cloud DB (Turso) is consistent with the stated cloud‑first design. Minor incoherence: the package uses cloud credentials (TURSO_URL/TURSO_TOKEN) and LLM endpoint URLs in its docs and CLI, but the registry metadata declares no required environment variables or primary credential — this mismatch reduces transparency.

Instruction Scope

Runtime instructions and code explicitly read local agent files (memory/YYYY‑MM‑DD.md, memory tree, metrics JSONL) and provide commands to auto‑ingest daily notes, consolidate, and 'sync-critical' hot+tree after conversations. The distiller and tree search can call external LLM endpoints; the sync writes data to cloud cold storage (Turso). These actions match the skill purpose but have broad scope: sensitive owner_profile and hot memory (names, family, events, tasks) may be uploaded or sent to LLMs. The SKILL.md and scripts also extract shell commands and file paths from text — useful for metadata but also capable of harvesting sensitive local paths/commands if invoked on arbitrary content.

Install Mechanism

No install spec (instruction-only) and pure Python scripts with no external package dependencies — low install risk. Nothing is downloaded from unknown URLs during install.

Credentials

The code expects external services (LLM endpoints, a Turso DB) and the README shows setting TURSO_URL/TURSO_TOKEN and LLM endpoints, but the skill metadata lists zero required env vars or credentials. This is a transparency gap: the skill will functionally require secrets/config to perform cloud sync and LLM calls, yet the registry did not declare them. Also metrics_tracker uses a WORKSPACE env var (with a fallback to a parent path) which may cause the tool to read or write outside expected directories if the environment is nonstandard.

Persistence & Privilege

always:false (normal) and user-invocable:true. The skill can be invoked autonomously (disable-model-invocation:false) which is normal for skills. Combine that with its documented behavior to 'sync critical' data to the cloud after every conversation and to call external LLMs: autonomous invocation plus network/cloud sync increases blast radius for data exfiltration if misconfigured. The skill does not declare that it will modify other skills or global config.

What to consider before installing

This skill appears to implement the memory system it promises, but there are important privacy and configuration issues you should consider before installing: - Credentials and endpoints: The README and CLI indicate you must provide a Turso DB URL/token and may supply LLM endpoints (HTTP). The registry metadata did not declare these required environment variables — verify and be prepared to provide secrets if you enable cloud sync or LLM features. - Data that may be uploaded: Hot memory explicitly contains owner identity/profile, family names, events, tasks and other personal context. The 'sync-critical' operation and cold storage (Turso) will transmit that data to the cloud. If you enable LLM distillation/search with external endpoints, portions of conversation text (and extracted metadata like URLs, commands, file paths) will be sent to that endpoint. - Principle of least privilege: If you decide to use this skill, prefer local-only operation first: avoid configuring an external Turso DB or remote LLM endpoint until you audit the code and are confident in the destination. Use localhost or a controlled internal LLM endpoint instead of public endpoints. - Audit the code paths: The code calls urllib.request.urlopen to user-supplied endpoints and subprocess to run memory_cli.py from metrics_tracker. Review/limit what files are passed into distillation/metadata extraction and where the workspace path points (WORKSPACE env var) to avoid accidental exposure of unrelated files. - Test safely: Run the scripts in a sandboxed environment with test data (no real PII) and without setting Turso/remote LLM configs. If you intend to enable cloud sync, inspect and, if necessary, redact sensitive fields before upload or add encryption/ACL protections on the destination DB. - If you need help deciding: ask the author (if known) for explicit declaration of required env vars and a clear privacy statement explaining exactly which fields get synced and how they are protected in transit and at rest. Given the declared functionality, these issues may be benign design choices, but the lack of declared credentials and the potential to send sensitive local context to external services are reasons to proceed cautiously.

Tiered Memory System v2.2.0

A mind that remembers everything is as useless as one that remembers nothing. The art is knowing what to keep. 🧠

EvoClaw-compatible three-tier memory system inspired by human cognition and PageIndex tree retrieval.

What's New in v2.2.0

🔄 Automatic Daily Note Ingestion

• Consolidation (daily/monthly/full modes) now auto-runs ingest-daily
• Bridges memory/YYYY-MM-DD.md files → tiered memory system
• No more manual ingestion required — facts flow automatically
• Fixes the "two disconnected data paths" problem

What's New in v2.1.0

🎯 Structured Metadata Extraction

• Automatic extraction of URLs, shell commands, and file paths from facts
• Preserved during distillation and consolidation
• Searchable by URL fragment

✅ Memory Completeness Validation

• Check daily notes for missing URLs, commands, and next steps
• Proactive warnings for incomplete information
• Actionable suggestions for improvement

🔍 Enhanced Search

• Search facts by URL fragment
• Get all stored URLs from warm memory
• Metadata-aware fact storage

🛡️ URL Preservation

• URLs explicitly preserved during LLM distillation
• Fallback metadata extraction if LLM misses them
• Command-line support for adding metadata manually

Architecture

┌─────────────────────────────────────────────────────┐
│              AGENT CONTEXT (~8-15KB)                │
│                                                     │
│  ┌──────────┐  ┌────────────────────────────────┐  │
│  │  Tree    │  │  Retrieved Memory Nodes         │  │
│  │  Index   │  │  (on-demand, 1-3KB)            │  │
│  │  (~2KB)  │  │                                │  │
│  │          │  │  Fetched per conversation      │  │
│  │  Always  │  │  based on tree reasoning       │  │
│  │  loaded  │  │                                │  │
│  └────┬─────┘  └────────────────────────────────┘  │
│       │                                             │
└───────┼─────────────────────────────────────────────┘
        │
        │ LLM-powered tree search
        │
┌───────▼─────────────────────────────────────────────┐
│              MEMORY TIERS                           │
│                                                     │
│  🔴 HOT (5KB)      🟡 WARM (50KB)     🟢 COLD (∞)  │
│                                                     │
│  Core memory       Scored facts      Full archive  │
│  - Identity        - 30-day         - Turso DB     │
│  - Owner profile   - Decaying       - Queryable    │
│  - Active context  - On-device      - 10-year      │
│  - Lessons (20 max)                                │
│                                                     │
│  Always in         Retrieved via     Retrieved via │
│  context           tree search       tree search   │
└─────────────────────────────────────────────────────┘

Design Principles

From Human Memory

• Consolidation — Short-term → long-term happens during consolidation cycles
• Relevance Decay — Unused memories fade; accessed memories strengthen
• Strategic Forgetting — Not remembering everything is a feature
• Hierarchical Organization — Navigate categories, not scan linearly

From PageIndex

• Vectorless Retrieval — LLM reasoning instead of embedding similarity
• Tree-Structured Index — O(log n) navigation, not O(n) scan
• Explainable Results — Every retrieval traces a path through categories
• Reasoning-Based Search — "Why relevant?" not "how similar?"

Cloud-First (EvoClaw)

• Device is replaceable — Soul lives in cloud (Turso)
• Critical sync — Hot + tree sync after every conversation
• Disaster recovery — Full restore in <2 minutes
• Multi-device — Same agent across phone/desktop/embedded

Memory Tiers

🔴 Hot Memory (5KB max)

Purpose: Core identity and active context, always in agent's context window.

Structure:

{
  "identity": {
    "agent_name": "Agent",
    "owner_name": "User",
    "owner_preferred_name": "User",
    "relationship_start": "2026-01-15",
    "trust_level": 0.95
  },
  "owner_profile": {
    "personality": "technical, direct communication",
    "family": ["Sarah (wife)", "Luna (daughter, 3yo)"],
    "topics_loved": ["AI architecture", "blockchain", "system design"],
    "topics_avoid": ["small talk about weather"],
    "timezone": "Australia/Sydney",
    "work_hours": "9am-6pm"
  },
  "active_context": {
    "projects": [
      {
        "name": "EvoClaw",
        "description": "Self-evolving agent framework",
        "status": "Active - BSC integration for hackathon"
      }
    ],
    "events": [
      {"text": "Hackathon deadline Feb 15", "timestamp": 1707350400}
    ],
    "tasks": [
      {"text": "Deploy to BSC testnet", "status": "pending", "timestamp": 1707350400}
    ]
  },
  "critical_lessons": [
    {
      "text": "Always test on testnet before mainnet",
      "category": "blockchain",
      "importance": 0.9,
      "timestamp": 1707350400
    }
  ]
}

Auto-pruning:

• Lessons: Max 20, removes lowest-importance when full
• Events: Keeps last 10 only
• Tasks: Max 10 pending
• Total size: Hard limit at 5KB, progressively prunes if exceeded

Generates: MEMORY.md — auto-rebuilt from structured hot state

🟡 Warm Memory (50KB max, 30-day retention)

Purpose: Recent distilled facts with decay scoring.

Entry format:

{
  "id": "abc123def456",
  "text": "Decided to use zero go-ethereum deps for EvoClaw to keep binary small",
  "category": "projects/evoclaw/architecture",
  "importance": 0.8,
  "created_at": 1707350400,
  "access_count": 3,
  "score": 0.742,
  "tier": "warm"
}

Scoring:

score = importance × recency_decay(age) × reinforcement(access_count)

recency_decay(age) = exp(-age_days / 30)
reinforcement(access) = 1 + 0.1 × access_count

Tier classification:

• score >= 0.7 → Hot (promote to hot state)
• score >= 0.3 → Warm (keep)
• score >= 0.05 → Cold (archive)
• score < 0.05 → Frozen (delete after retention period)

Eviction triggers:

1. Age > 30 days AND score < 0.3
2. Total warm size > 50KB (evicts lowest-scored)
3. Manual consolidation

🟢 Cold Memory (Unlimited, Turso)

Purpose: Long-term archive, queryable but never bulk-loaded.

Schema:

CREATE TABLE cold_memories (
  id TEXT PRIMARY KEY,
  agent_id TEXT NOT NULL,
  text TEXT NOT NULL,
  category TEXT NOT NULL,
  importance REAL DEFAULT 0.5,
  created_at INTEGER NOT NULL,
  access_count INTEGER DEFAULT 0
);

CREATE TABLE critical_state (
  agent_id TEXT PRIMARY KEY,
  data TEXT NOT NULL,  -- {hot_state, tree_nodes, timestamp}
  updated_at INTEGER NOT NULL
);

Retention: 10 years (configurable) Cleanup: Monthly consolidation removes frozen entries older than retention period

Tree Index

Purpose: Hierarchical category map for O(log n) retrieval.

Constraints:

• Max 50 nodes
• Max depth 4 levels
• Max 2KB serialized
• Max 10 children per node

Example:

Memory Tree Index
==================================================
📂 Root (warm:15, cold:234)
  📁 owner — Owner profile and preferences
     Memories: warm=5, cold=89
  📁 projects — Active projects
     Memories: warm=8, cold=67
    📁 projects/evoclaw — EvoClaw framework
       Memories: warm=6, cold=45
      📁 projects/evoclaw/bsc — BSC integration
         Memories: warm=3, cold=12
  📁 technical — Technical setup and config
     Memories: warm=2, cold=34
  📁 lessons — Learned lessons and rules
     Memories: warm=0, cold=44

Nodes: 7/50
Size: 1842 / 2048 bytes

Operations:

• --add PATH DESC — Add category node
• --remove PATH — Remove node (only if no data)
• --prune — Remove dead nodes (no activity in 60+ days)
• --show — Pretty-print tree

Distillation Engine

Purpose: Three-stage compression of conversations.

Pipeline:

Raw conversation (500B)
  ↓ Stage 1→2: Extract structured info
Distilled fact (80B)
  ↓ Stage 2→3: Generate one-line summary
Core summary (20B)

Stage 1→2: Raw → Distilled

Input: Raw conversation text Output: Structured JSON

{
  "fact": "User decided to use raw JSON-RPC for BSC to avoid go-ethereum dependency",
  "emotion": "determined",
  "people": ["User"],
  "topics": ["blockchain", "architecture", "dependencies"],
  "actions": ["decided to use raw JSON-RPC", "avoid go-ethereum"],
  "outcome": "positive"
}

Modes:

• rule: Regex/heuristic extraction (fast, no LLM)
• llm: LLM-powered extraction (accurate, requires endpoint)

Usage:

# Rule-based (default)
distiller.py --text "Had a productive chat about the BSC integration..." --mode rule

# LLM-powered
distiller.py --text "..." --mode llm --llm-endpoint http://localhost:8080/complete

# With core summary
distiller.py --text "..." --mode rule --core-summary

Stage 2→3: Distilled → Core Summary

Purpose: One-line summary for tree index

Example:

Distilled: {
  "fact": "User decided raw JSON-RPC for BSC, no go-ethereum",
  "outcome": "positive"
}

Core summary: "BSC integration: raw JSON-RPC (no deps)"

Target: <30 bytes

LLM-Powered Tree Search

Purpose: Semantic search through tree structure using LLM reasoning.

How it works:

1. Build prompt with tree structure + query
2. LLM reasons about which categories are relevant
3. Returns category paths with relevance scores
4. Fetches memories from those categories

Example:

Query: "What did we decide about the hackathon deadline?"

Keyword search returns:

• projects/evoclaw (0.8)
• technical/deployment (0.4)

LLM search reasons:

• projects/evoclaw/bsc (0.95) — "BSC integration for hackathon"
• active_context/events (0.85) — "Deadline mentioned here"

LLM prompt template:

You are a memory retrieval system. Given a memory tree index and a query, 
identify which categories are relevant.

Memory Tree Index:
  projects/evoclaw — EvoClaw framework (warm:6, cold:45)
  projects/evoclaw/bsc — BSC integration (warm:3, cold:12)
  ...

User Query: What did we decide about the hackathon deadline?

Output (JSON):
[
  {"path": "projects/evoclaw/bsc", "relevance": 0.95, "reason": "BSC work for hackathon"},
  {"path": "active_context/events", "relevance": 0.85, "reason": "deadline tracking"}
]

Usage:

# Keyword search (fast)
tree_search.py --query "BSC integration" --tree-file memory-tree.json --mode keyword

# LLM search (accurate)
tree_search.py --query "what did we decide about hackathon?" \
  --tree-file memory-tree.json --mode llm --llm-endpoint http://localhost:8080/complete

# Generate prompt for external LLM
tree_search.py --query "..." --tree-file memory-tree.json \
  --mode llm --llm-prompt-file prompt.txt

Multi-Agent Support

Agent ID scoping — All operations support --agent-id flag.

File layout:

memory/
  default/
    warm-memory.json
    memory-tree.json
    hot-memory-state.json
    metrics.json
  agent-2/
    warm-memory.json
    memory-tree.json
    ...
MEMORY.md              # default agent
MEMORY-agent-2.md      # agent-2

Cold storage: Agent-scoped queries via agent_id column

Usage:

# Store for agent-2
memory_cli.py store --text "..." --category "..." --agent-id agent-2

# Retrieve for agent-2
memory_cli.py retrieve --query "..." --agent-id agent-2

# Consolidate agent-2
memory_cli.py consolidate --mode daily --agent-id agent-2

Consolidation Modes

Purpose: Periodic memory maintenance and optimization.

Quick (hourly)

• Warm eviction (score-based)
• Archive expired to cold
• Recalculate all scores
• Rebuild MEMORY.md

Daily

• Everything in Quick
• Tree prune (remove dead nodes, 60+ days no activity)

Monthly

• Everything in Daily
• Tree rebuild (LLM-powered restructuring, future)
• Cold cleanup (delete frozen entries older than retention)

Full

• Everything in Monthly
• Full recalculation of all scores
• Deep tree analysis
• Generate consolidation report

Usage:

# Quick consolidation (default)
memory_cli.py consolidate

# Daily (run via cron)
memory_cli.py consolidate --mode daily

# Monthly (run via cron)
memory_cli.py consolidate --mode monthly --db-url "$TURSO_URL" --auth-token "$TURSO_TOKEN"

Recommended schedule:

• Quick: Every 2-4 hours (heartbeat)
• Daily: Midnight via cron
• Monthly: 1st of month via cron

Critical Sync (Cloud-First)

Purpose: Cloud backup of hot state + tree after every conversation.

What syncs:

• Hot memory state (identity, owner profile, active context, lessons)
• Tree index (structure + counts)
• Timestamp

Recovery: If device lost, restore from cloud in <2 minutes

Usage:

# Manual critical sync
memory_cli.py sync-critical --db-url "$TURSO_URL" --auth-token "$TURSO_TOKEN" --agent-id default

# Automatic: Call after every important conversation
# In agent code:
#   1. Process conversation
#   2. Store distilled facts
#   3. Call sync-critical

Retry strategy: Exponential backoff if cloud unreachable (5s, 10s, 20s, 40s)

Metrics & Observability

Tracked metrics:

{
  "tree_index_size_bytes": 1842,
  "tree_node_count": 37,
  "hot_memory_size_bytes": 4200,
  "warm_memory_count": 145,
  "warm_memory_size_kb": 38.2,
  "retrieval_count": 234,
  "evictions_today": 12,
  "reinforcements_today": 67,
  "consolidation_count": 8,
  "last_consolidation": 1707350400,
  "context_tokens_saved": 47800,
  "timestamp": "2026-02-10T14:30:00"
}

Usage:

memory_cli.py metrics --agent-id default

Key metrics:

• context_tokens_saved — Estimated tokens saved vs. flat MEMORY.md
• retrieval_count — How often memories are accessed
• evictions_today — Memory pressure indicator
• warm_memory_size_kb — Storage usage

Commands Reference

Store

memory_cli.py store --text "Fact text" --category "path/to/category" [--importance 0.8] [--agent-id default]

Importance guide:

• 0.9-1.0 — Critical decisions, credentials, core identity
• 0.7-0.8 — Project decisions, architecture, preferences
• 0.5-0.6 — General facts, daily events
• 0.3-0.4 — Casual mentions, low priority

Example:

memory_cli.py store \
  --text "Decided to deploy EvoClaw on BSC testnet before mainnet" \
  --category "projects/evoclaw/deployment" \
  --importance 0.85 \
  --db-url "$TURSO_URL" --auth-token "$TURSO_TOKEN"

# Store with explicit metadata (v2.1.0+)
memory_cli.py store \
  --text "Z-Image ComfyUI model for photorealistic images" \
  --category "tools/image-generation" \
  --importance 0.8 \
  --url "https://docs.comfy.org/tutorials/image/z-image/z-image" \
  --command "huggingface-cli download Tongyi-MAI/Z-Image" \
  --path "/home/user/models/"

Validate (v2.1.0)

memory_cli.py validate [--file PATH] [--agent-id default]

Purpose: Check daily notes for incomplete information (missing URLs, commands, next steps).

Example:

# Validate today's daily notes
memory_cli.py validate

# Validate specific file
memory_cli.py validate --file memory/2026-02-13.md

Output:

{
  "status": "warning",
  "warnings_count": 2,
  "warnings": [
    "Tool 'Z-Image' mentioned without URL/documentation link",
    "Action 'install' mentioned without command example"
  ],
  "suggestions": [
    "Add URLs for mentioned tools/services",
    "Include command examples for setup/installation steps",
    "Document next steps after decisions"
  ]
}

Extract Metadata (v2.1.0)

memory_cli.py extract-metadata --file PATH

Purpose: Extract structured metadata (URLs, commands, paths) from a file.

Example:

memory_cli.py extract-metadata --file memory/2026-02-13.md

Output:

{
  "file": "memory/2026-02-13.md",
  "metadata": {
    "urls": [
      "https://docs.comfy.org/tutorials/image/z-image/z-image",
      "https://github.com/Lightricks/LTX-Video"
    ],
    "commands": [
      "huggingface-cli download Tongyi-MAI/Z-Image",
      "git clone https://github.com/Lightricks/LTX-Video.git"
    ],
    "paths": [
      "/home/peter/ai-stack/comfyui/models",
      "./configs/ltx-video-2-config.yaml"
    ]
  },
  "summary": {
    "urls_count": 2,
    "commands_count": 2,
    "paths_count": 2
  }
}

Search by URL (v2.1.0)

memory_cli.py search-url --url FRAGMENT [--limit 5] [--agent-id default]

Purpose: Search facts by URL fragment.

Example:

# Find all facts with comfy.org URLs
memory_cli.py search-url --url "comfy.org"

# Find GitHub repos
memory_cli.py search-url --url "github.com" --limit 10

Output:

{
  "query": "comfy.org",
  "results_count": 1,
  "results": [
    {
      "id": "abc123",
      "text": "Z-Image ComfyUI model for photorealistic images",
      "category": "tools/image-generation",
      "metadata": {
        "urls": ["https://docs.comfy.org/tutorials/image/z-image/z-image"],
        "commands": ["huggingface-cli download Tongyi-MAI/Z-Image"],
        "paths": []
      }
    }
  ]
}

Retrieve

memory_cli.py retrieve --query "search query" [--limit 5] [--llm] [--llm-endpoint URL] [--agent-id default]

Modes:

• Default: Keyword-based tree + warm + cold search
• --llm: LLM-powered semantic tree search

Example:

# Keyword search
memory_cli.py retrieve --query "BSC deployment decision" --limit 5

# LLM search (more accurate)
memory_cli.py retrieve \
  --query "what did we decide about blockchain integration?" \
  --llm --llm-endpoint http://localhost:8080/complete \
  --db-url "$TURSO_URL" --auth-token "$TURSO_TOKEN"

Distill

memory_cli.py distill --text "raw conversation" [--llm] [--llm-endpoint URL]

Example:

# Rule-based distillation
memory_cli.py distill --text "User: Let's deploy to testnet first. Agent: Good idea, safer that way."

# LLM distillation
memory_cli.py distill \
  --text "Long conversation with nuance..." \
  --llm --llm-endpoint http://localhost:8080/complete

Output:

{
  "distilled": {
    "fact": "Decided to deploy to testnet before mainnet",
    "emotion": "cautious",
    "people": [],
    "topics": ["deployment", "testnet", "safety"],
    "actions": ["deploy to testnet"],
    "outcome": "positive"
  },
  "mode": "rule",
  "original_size": 87,
  "distilled_size": 156
}

Hot Memory

# Update hot state
memory_cli.py hot --update KEY JSON [--agent-id default]

# Rebuild MEMORY.md
memory_cli.py hot --rebuild [--agent-id default]

# Show current hot state
memory_cli.py hot [--agent-id default]

Keys:

• identity — Agent/owner identity info
• owner_profile — Owner preferences, personality
• lesson — Add critical lesson
• event — Add event to active context
• task — Add task to active context
• project — Add/update project

Examples:

# Update owner profile
memory_cli.py hot --update owner_profile '{"timezone": "Australia/Sydney", "work_hours": "9am-6pm"}'

# Add lesson
memory_cli.py hot --update lesson '{"text": "Always test on testnet first", "category": "blockchain", "importance": 0.9}'

# Add project
memory_cli.py hot --update project '{"name": "EvoClaw", "status": "Active", "description": "Self-evolving agent framework"}'

# Rebuild MEMORY.md
memory_cli.py hot --rebuild

Tree

# Show tree
memory_cli.py tree --show [--agent-id default]

# Add node
memory_cli.py tree --add "path/to/category" "Description" [--agent-id default]

# Remove node
memory_cli.py tree --remove "path/to/category" [--agent-id default]

# Prune dead nodes
memory_cli.py tree --prune [--agent-id default]

Examples:

# Add category
memory_cli.py tree --add "projects/evoclaw/bsc" "BSC blockchain integration"

# Remove empty category
memory_cli.py tree --remove "old/unused/path"

# Prune dead nodes (60+ days no activity)
memory_cli.py tree --prune

Cold Storage

# Initialize Turso tables
memory_cli.py cold --init --db-url URL --auth-token TOKEN

# Query cold storage
memory_cli.py cold --query "search term" [--limit 10] [--agent-id default] --db-url URL --auth-token TOKEN

Examples:

# Init tables (once)
memory_cli.py cold --init --db-url "https://your-db.turso.io" --auth-token "your-token"

# Query cold archive
memory_cli.py cold --query "blockchain decision" --limit 10 --db-url "$TURSO_URL" --auth-token "$TURSO_TOKEN"

Configuration

File: config.json (optional, uses defaults if not present)

{
  "agent_id": "default",
  "hot": {
    "max_bytes": 5120,
    "max_lessons": 20,
    "max_events": 10,
    "max_tasks": 10
  },
  "warm": {
    "max_kb": 50,
    "retention_days": 30,
    "eviction_threshold": 0.3
  },
  "cold": {
    "backend": "turso",
    "retention_years": 10
  },
  "scoring": {
    "half_life_days": 30,
    "reinforcement_boost": 0.1
  },
  "tree": {
    "max_nodes": 50,
    "max_depth": 4,
    "max_size_bytes": 2048
  },
  "distillation": {
    "aggression": 0.7,
    "max_distilled_bytes": 100,
    "mode": "rule"
  },
  "consolidation": {
    "warm_eviction": "hourly",
    "tree_prune": "daily",
    "tree_rebuild": "monthly"
  }
}

Integration with OpenClaw Agents

After Conversation

import subprocess
import json

def process_conversation(user_message, agent_response, category="conversations"):
    # 1. Distill conversation
    text = f"User: {user_message}\nAgent: {agent_response}"
    result = subprocess.run(
        ["python3", "skills/tiered-memory/scripts/memory_cli.py", "distill", "--text", text],
        capture_output=True, text=True
    )
    distilled = json.loads(result.stdout)
    
    # 2. Determine importance
    importance = 0.7 if "decision" in distilled["distilled"]["outcome"] else 0.5
    
    # 3. Store
    subprocess.run([
        "python3", "skills/tiered-memory/scripts/memory_cli.py", "store",
        "--text", distilled["distilled"]["fact"],
        "--category", category,
        "--importance", str(importance),
        "--db-url", os.getenv("TURSO_URL"),
        "--auth-token", os.getenv("TURSO_TOKEN")
    ])
    
    # 4. Critical sync
    subprocess.run([
        "python3", "skills/tiered-memory/scripts/memory_cli.py", "sync-critical",
        "--db-url", os.getenv("TURSO_URL"),
        "--auth-token", os.getenv("TURSO_TOKEN")
    ])

Before Responding (Retrieval)

def get_relevant_context(query):
    result = subprocess.run(
        [
            "python3", "skills/tiered-memory/scripts/memory_cli.py", "retrieve",
            "--query", query,
            "--limit", "5",
            "--llm",
            "--llm-endpoint", "http://localhost:8080/complete",
            "--db-url", os.getenv("TURSO_URL"),
            "--auth-token", os.getenv("TURSO_TOKEN")
        ],
        capture_output=True, text=True
    )
    
    memories = json.loads(result.stdout)
    return "\n".join([f"- {m['text']}" for m in memories])

Heartbeat Consolidation

import schedule

# Hourly quick consolidation
schedule.every(2).hours.do(lambda: subprocess.run([
    "python3", "skills/tiered-memory/scripts/memory_cli.py", "consolidate",
    "--mode", "quick",
    "--db-url", os.getenv("TURSO_URL"),
    "--auth-token", os.getenv("TURSO_TOKEN")
]))

# Daily tree prune
schedule.every().day.at("00:00").do(lambda: subprocess.run([
    "python3", "skills/tiered-memory/scripts/memory_cli.py", "consolidate",
    "--mode", "daily",
    "--db-url", os.getenv("TURSO_URL"),
    "--auth-token", os.getenv("TURSO_TOKEN")
]))

# Monthly full consolidation
schedule.every().month.do(lambda: subprocess.run([
    "python3", "skills/tiered-memory/scripts/memory_cli.py", "consolidate",
    "--mode", "monthly",
    "--db-url", os.getenv("TURSO_URL"),
    "--auth-token", os.getenv("TURSO_TOKEN")
]))

LLM Integration

Model Recommendations

For Distillation & Tree Search:

• Claude 3 Haiku (fast, cheap, excellent structure)
• GPT-4o-mini (good balance)
• Gemini 1.5 Flash (very fast)

For Tree Rebuilding:

• Claude 3.5 Sonnet (better reasoning)
• GPT-4o (strong planning)

Cost Optimization

1. Use cheaper models for frequent operations (distill, search)
2. Batch distillation — Queue conversations, distill in batch
3. Cache tree prompts — Tree structure doesn't change often
4. Skip LLM for simple — Use rule-based for short conversations

Example LLM Endpoint

from flask import Flask, request, jsonify

app = Flask(__name__)

@app.route("/complete", methods=["POST"])
def complete():
    data = request.json
    prompt = data["prompt"]
    
    # Call your LLM (OpenAI, Anthropic, local model, etc.)
    response = llm_client.complete(prompt)
    
    return jsonify({"text": response})

if __name__ == "__main__":
    app.run(port=8080)

Performance Characteristics

Context Size:

• Hot: ~5KB (always loaded)
• Tree: ~2KB (always loaded)
• Retrieved: ~1-3KB per query
• Total: ~8-15KB (constant, regardless of agent age)

Retrieval Speed:

• Keyword: 10-20ms
• LLM tree search: 300-600ms
• Cold query: 50-100ms

5-Year Scenario:

• Hot: Still 5KB (living document)
• Warm: Last 30 days (~50KB)
• Cold: ~50MB in Turso (compressed distilled facts)
• Tree: Still 2KB (different nodes, same size)
• Context per session: Same as day 1

Comparison with Alternatives

System	Memory Model	Scaling	Accuracy	Cost
Flat MEMORY.md	Linear text	❌ Months	⚠️ Degrades	❌ Linear
Vector RAG	Embeddings	✅ Years	⚠️ Similarity≠relevance	⚠️ Moderate
EvoClaw Tiered	Tree + tiers	✅ Decades	✅ Reasoning-based	✅ Fixed

Why tree > vectors:

• Accuracy: 98%+ vs. 70-80% (PageIndex benchmark)
• Explainable: "Projects → EvoClaw → BSC" vs. "cosine 0.73"
• Multi-hop: Natural vs. poor
• False positives: Low vs. high

Troubleshooting

Tree size exceeding limit

# Prune dead nodes
memory_cli.py tree --prune

# Check which nodes are largest
memory_cli.py tree --show | grep "Memories:"

# Manually remove unused categories
memory_cli.py tree --remove "unused/category"

Warm memory filling up

# Run consolidation
memory_cli.py consolidate --mode daily --db-url "$TURSO_URL" --auth-token "$TURSO_TOKEN"

# Check stats
memory_cli.py metrics

# Lower eviction threshold (keeps less in warm)
# Edit config.json: "eviction_threshold": 0.4

Hot memory exceeding 5KB

# Hot auto-prunes, but check structure
memory_cli.py hot

# Remove old projects/tasks manually
memory_cli.py hot --update project '{"name": "OldProject", "status": "Completed"}'

# Rebuild to force pruning
memory_cli.py hot --rebuild

LLM search failing

# Fallback to keyword search (automatic)
memory_cli.py retrieve --query "..." --limit 5

# Test LLM endpoint
curl -X POST http://localhost:8080/complete -d '{"prompt": "test"}'

# Generate prompt for external testing
tree_search.py --query "..." --tree-file memory/memory-tree.json --mode llm --llm-prompt-file test.txt

Migration from v1.x

Backward compatible: Existing warm-memory.json and memory-tree.json files work as-is.

New files:

• config.json (optional, uses defaults)
• hot-memory-state.json (auto-created)
• metrics.json (auto-created)

Steps:

1. Update skill: clawhub update tiered-memory
2. Run consolidation to rebuild hot state: memory_cli.py consolidate
3. Initialize cold storage (optional): memory_cli.py cold --init --db-url ... --auth-token ...
4. Configure agent to use new commands (see Integration section)

Migration from v2.0 to v2.1

Fully backward compatible: Existing memory files work without changes.

What's new:

• ✅ Metadata automatically extracted from existing facts when loaded
• ✅ New commands: validate, extract-metadata, search-url
• ✅ store command now accepts --url, --command, --path flags
• ✅ Distillation preserves URLs and technical details
• ✅ No action required - just update and use new features

Testing the upgrade:

# Update skill
clawhub update tiered-memory

# Test metadata extraction
memory_cli.py extract-metadata --file memory/2026-02-13.md

# Validate your recent notes
memory_cli.py validate

# Search by URL
memory_cli.py search-url --url "github.com"

References

• Design: /docs/TIERED-MEMORY.md (EvoClaw)
• Cloud Sync: /docs/CLOUD-SYNC.md (EvoClaw)
• Inspiration: PageIndex (tree-based retrieval)

---

v2.1.0 — A mind that remembers everything is as useless as one that remembers nothing. The art is knowing what to keep. Now with structured metadata to remember HOW, not just WHAT. 🧠🌲🔗