Vta Memory

Reward and motivation system for AI agents. Dopamine-like wanting, not just doing. Part of the AI Brain series.

Security Scan

The skill's code, requirements, and runtime instructions line up with a local reward/motivation system for agents; it primarily reads/writes files in the agent workspace and does not request external credentials or hidden network endpoints, but it will read conversation transcripts and auto-inject a first‑person state file into sessions (privacy/behavioral impact to consider).

Purpose & Capability

The name/description (VTA reward/motivation) match what the repository implements: local state file, reward logging, seeking/anticipation, scheduled decay, and a dashboard. Required binaries (jq, awk, bc) are used throughout the scripts and are appropriate.

Instruction Scope

Instructions and scripts read session transcripts (~/.openclaw/agents/$AGENT_ID/sessions) to extract reward signals, write multiple JSON/MD files into ~/.openclaw/workspace/memory, and generate VTA_STATE.md which OpenClaw auto-injects into future sessions. This is coherent with purpose but has privacy/behavior implications: it ingests conversation history and summarizes it into content that influences agent behavior (first‑person narrative).

Install Mechanism

There is no remote install that downloads arbitrary code; included install.sh and local scripts are executed. install.sh optionally uses the local 'openclaw' CLI to register cron jobs; if openclaw is absent it prints the commands for manual setup. No external download URLs or extractable archives are present.

Credentials

No environment variables or external credentials are requested. The workspace and agent transcript paths accessed are proportional to the stated goal (detecting rewards from conversation).

Persistence & Privilege

The skill writes VTA_STATE.md into the workspace which OpenClaw will auto-inject into sessions — deliberately influencing agent outputs. Cron jobs for periodic processing are optional (install.sh only registers them when run with --with-cron). The skill does not declare always:true; autonomous agent invocation is possible via cron or agent-turns but not inherently granted by the skill itself.

Assessment

This skill appears internally consistent and runs entirely on local files, but review these before installing: 1) It reads conversation transcripts (~/.openclaw/agents/<id>/sessions) and writes summaries and logs into ~/.openclaw/workspace/memory — sensitive conversation content will be processed and stored locally. 2) It generates VTA_STATE.md (first‑person motivation text) which OpenClaw auto-injects into sessions and will change agent behavior; test it in a non-production agent first. 3) install.sh can register cron agent-turns via the openclaw CLI to run decay/encoding periodically — run install.sh without --with-cron initially to inspect behavior, and only enable cron after confirming you accept automated runs. 4) Inspect workspace sync/backup settings (if your workspace is backed up to cloud or shared) because processed transcripts and logs are stored in cleartext. If you want to proceed: run in a sandbox/test agent, audit the generated files (reward-state.json, brain-events.jsonl, VTA_STATE.md) and confirm no network calls are being made from your environment.

Reward and motivation for AI agents. Part of the AI Brain series.

Give your AI agent genuine wanting — not just doing things when asked, but having drive, seeking rewards, and looking forward to things.

The Problem

Current AI agents:

• ✅ Do what they're asked
• ❌ Don't want anything
• ❌ Have no internal motivation
• ❌ Don't feel satisfaction from accomplishment

Without a reward system, there's no desire. Just execution.

The Solution

Track motivation through:

• Drive — overall motivation level (0-1)
• Rewards — logged accomplishments that boost drive
• Seeking — what I actively want more of
• Anticipation — what I'm looking forward to

Quick Start

1. Install

cd ~/.openclaw/workspace/skills/vta-memory
./install.sh --with-cron

This will:

• Create memory/reward-state.json
• Generate VTA_STATE.md (auto-injected into sessions!)
• Set up cron for drive decay every 8 hours

2. Check motivation

./scripts/load-motivation.sh

# ⭐ Current Motivation State:
# Drive level: 0.73 (motivated — ready to work)
# Seeking: creative work, building brain skills
# Looking forward to: showing my work

3. Log rewards

./scripts/log-reward.sh --type accomplishment --source "finished the feature" --intensity 0.8

# ⭐ Reward logged!
#    Type: accomplishment
#    Drive: 0.50 → 0.66 (+0.16)

4. Add anticipation

./scripts/anticipate.sh --add "morning conversation"

# ⭐ Now looking forward to: morning conversation
#    Drive: 0.66 → 0.71 (+0.05)

Scripts

Script	Purpose
install.sh	Set up vta-memory (run once)
get-drive.sh	Read current motivation state
log-reward.sh	Log a reward, boost drive
load-motivation.sh	Human-readable for session context
decay-drive.sh	Drive fades without rewards
anticipate.sh	Add/remove things to look forward to
seek.sh	Add/remove things we're actively seeking
sync-motivation.sh	Generate VTA_STATE.md for auto-injection
resolve-anticipation.sh	Mark anticipation as fulfilled
update-watermark.sh	Update processing watermark
preprocess-rewards.sh	Extract reward signals from transcript
generate-dashboard.sh	Generate unified brain dashboard HTML

Brain Dashboard

Visual dashboard showing all installed brain skills:

./scripts/generate-dashboard.sh
open ~/.openclaw/workspace/brain-dashboard.html

• Auto-detects other brain skills (hippocampus, amygdala)
• Shows install prompts for missing skills
• Regenerates on install and cron runs
• Reads avatar/name from IDENTITY.md

After Installation (for agents)

When you install this skill, notify the user about the dashboard:

⭐ VTA installed!

View your agent's drive level in the Brain Dashboard: ~/.openclaw/workspace/brain-dashboard.html

Reward Types

Type	When to Use
accomplishment	Completed a task, shipped something
social	User appreciation, "thanks!", positive feedback
curiosity	Learned something new, discovered interesting info
connection	Deep conversation, bonding moment
creative	Made something, expressed creativity
competence	Solved a hard problem, did something well

How Drive Works

Rewards Boost Drive

drive_boost = intensity × 0.2
new_drive = min(current + boost, 1.0)

A high-intensity (0.9) reward boosts drive by 0.18.

Anticipation Boosts Drive

Looking forward to something adds +0.05 to drive.

Drive Decays Without Rewards

# Every 8 hours (via cron)
new_drive = current + (baseline - current) × 0.15

Without rewards, motivation fades toward baseline (0.5).

Auto-Injection

After install, VTA_STATE.md is created in your workspace root.

OpenClaw automatically injects all *.md files from workspace into session context:

1. New session starts
2. VTA_STATE.md is auto-loaded
3. You see your motivation state
4. Behavior influenced by drive level

How Drive Affects Behavior

Drive Level	Description	Behavior
> 0.8	Highly motivated	Eager, proactive, take on challenges
0.6 - 0.8	Motivated	Ready to work, engaged
0.4 - 0.6	Moderate	Can engage but not pushing
0.2 - 0.4	Low	Prefer simple tasks, need a win
< 0.2	Very low	Unmotivated, need rewards to get going

State File Format

{
  "drive": 0.73,
  "baseline": { "drive": 0.5 },
  "seeking": ["creative work", "building brain skills"],
  "anticipating": ["morning conversation"],
  "recentRewards": [
    {
      "type": "creative",
      "source": "built VTA reward system",
      "intensity": 0.9,
      "boost": 0.18,
      "timestamp": "2026-02-01T03:25:00Z"
    }
  ],
  "rewardHistory": {
    "totalRewards": 1,
    "byType": { "creative": 1, ... }
  }
}

Event Logging

Track motivation patterns over time:

# Log encoding run
./scripts/log-event.sh encoding rewards_found=2 drive=0.65

# Log decay
./scripts/log-event.sh decay drive_before=0.6 drive_after=0.53

# Log reward
./scripts/log-event.sh reward type=accomplishment intensity=0.8

Events append to ~/.openclaw/workspace/memory/brain-events.jsonl:

{"ts":"2026-02-11T10:45:00Z","type":"vta","event":"encoding","rewards_found":2,"drive":0.65}

Use for analyzing motivation cycles — when does drive peak? What rewards work best?

AI Brain Series

Part	Function	Status
hippocampus	Memory formation, decay, reinforcement	✅ Live
amygdala-memory	Emotional processing	✅ Live
basal-ganglia-memory	Habit formation	🚧 Development
anterior-cingulate-memory	Conflict detection	🚧 Development
insula-memory	Internal state awareness	🚧 Development
vta-memory	Reward and motivation	✅ Live

Philosophy: Wanting vs Doing

The VTA produces dopamine — not the "pleasure chemical" but the "wanting chemical."

Neuroscience distinguishes:

• Wanting (motivation) — drive toward something
• Liking (pleasure) — enjoyment when you get it

You can want something you don't like (addiction) or like something you don't want (guilty pleasures).

This skill implements wanting — the drive that makes action happen. Without it, why would an AI do anything beyond what it's explicitly asked?

---

Built with ⭐ by the OpenClaw community