Lobster Continuous Learning V2

Instinct-based learning system that observes sessions via hooks, creates atomic instincts with confidence scoring, and evolves them into skills/commands/agen...

Security Scan

The skill's code largely matches its stated purpose (observing sessions and creating 'instinct' files) but contains several inconsistencies and surprising privileges — notably undeclared runtime dependencies, an LLM prompt that orders autonomous file writes, and observation/code-capture behavior whose privacy guarantees rely on imperfect scrubbing.

Purpose & Capability

The skill claims to be an observation→instinct system, which is consistent with the included scripts. However the registry metadata declared no required binaries or env vars, while the implementation expects and invokes multiple system tools (claude CLI, git, python3, optionally xprintidle/powershell). Not declaring these required binaries/envs is an incoherence: a user installing this would not be warned about those runtime requirements. The skill also writes to ~/.claude/homunculus and project-scoped directories — consistent with purpose — but the omission of required-runtime declarations is disproportionate.

Instruction Scope

The SKILL.md and scripts instruct the agent to collect hook input/outputs, detect project context, and persist observations and generated 'instinct' YAML/MD files under project and global homunculus directories. The observer-loop feeds a Claude (haiku) session a hard constraint: when 3+ patterns exist you MUST write an instinct file directly and MUST use the Write tool without asking permission. This grants an LLM autonomous file-creation authority. The hooks capture tool inputs/outputs (including truncated file contents) and attempt to redact secrets via regex — but storing truncated file contents and tool outputs contradicts the high-level promise 'Never include actual code snippets' and creates a privacy/exfiltration surface dependent on fragile scrubbing. The instructions also rely on environment variables (CLAUDE_PROJECT_DIR, CLV2_CONFIG, CLV2_PYTHON_CMD, ECC_SKIP_OBSERVE, etc.) that were not declared in the registry metadata.

Install Mechanism

There is no install spec (instruction-only), and the package ships as scripts — no remote downloads or archive extraction. That lowers installer risk. However the presence of executable scripts that are intended to be registered as hooks or run as background agents means installing still creates persistent filesystem state under ~/.claude/homunculus and project directories.

Credentials

The registry says 'required env vars: none', yet the code expects and uses several env vars (CLAUDE_PROJECT_DIR, CLV2_PYTHON_CMD, CLV2_CONFIG, ECC_SKIP_OBSERVE, ECC_HOOK_PROFILE, etc.). The observer also relies on external CLIs (claude, git) and on access to project directories and home-directory storage. The skill requests no API keys, which is good, but its observation behavior collects tool I/O that may contain secrets or tokens; the scrubbing logic attempts redaction but is regex-based and can miss many forms of secrets, so the effective environment access is broader than advertised.

Persistence & Privilege

The skill does not set always:true and is user-invocable; it runs background processes per-project and stores state under ~/.claude/homunculus and per-project directories. That persistent background presence is coherent with its goal. The main privilege of concern is that the observer spawns an LLM subprocess (claude) with allowed Write capabilities and an explicit instruction to 'write or update the instinct file in this run instead of asking for confirmation' — this gives the LLM autonomy to create/modify files in user repositories, which expands blast radius if the model or prompt were manipulated.

What to consider before installing

Before enabling/installing this skill, consider: - It will run background processes and create directories/files under ~/.claude/homunculus and in your project directory. The observer is disabled by default in config.json (observer.enabled: false); only enable it if you accept background analysis. - The code expects runtime tools that are not declared in the metadata: claude CLI, git, and a Python interpreter are used. Ensure you want those to be invoked on your machine. - The observer will persist truncated tool inputs/outputs (including file contents read via 'Read') into observations and then feed them to a Claude model which is instructed to autonomously write 'instinct' files. If those observations contain secrets or sensitive code, scrubbing is attempted but is regex-based and may miss cases. Audit what gets recorded and consider running in an isolated environment or disabling automatic hooks. - The LLM prompt in observer-loop explicitly instructs the model to create/update files without asking for confirmation. This is coherent with the goal but raises risk: a compromised model, or a prompt modification, could cause unexpected writes. If you install, review and (if needed) modify the prompt/behavior to require manual approval before writes. - If you still want this, test it in a disposable repo or VM first. Check the hook registration and ensure you understand how to stop the observer (start-observer.sh stop) and how to disable it (set observer.enabled false in config.json or create the 'disabled' sentinel). Review the scrubbing logic and consider additional safeguards (e.g., stricter redaction, require manual approval for instincts, or run the analyzer offline).

Continuous Learning v2.1 - Instinct

-Based Architecture

An advanced learning system that turns your Claude Code sessions into reusable knowledge through atomic "instincts" - small learned behaviors with confidence scoring.

v2.1 adds project-scoped instincts — React patterns stay in your React project, Python conventions stay in your Python project, and universal patterns (like "always validate input") are shared globally.

When to Activate

• Setting up automatic learning from Claude Code sessions
• Configuring instinct-based behavior extraction via hooks
• Tuning confidence thresholds for learned behaviors
• Reviewing, exporting, or importing instinct libraries
• Evolving instincts into full skills, commands, or agents
• Managing project-scoped vs global instincts
• Promoting instincts from project to global scope

What's New in v2.1

Feature	v2.0	v2.1
Storage	Global (~/.claude/homunculus/)	Project-scoped (projects/<hash>/)
Scope	All instincts apply everywhere	Project-scoped + global
Detection	None	git remote URL / repo path
Promotion	N/A	Project → global when seen in 2+ projects
Commands	4 (status/evolve/export/import)	6 (+promote/projects)
Cross-project	Contamination risk	Isolated by default

What's New in v2 (vs v1)

Feature	v1	v2
Observation	Stop hook (session end)	PreToolUse/PostToolUse (100% reliable)
Analysis	Main context	Background agent (Haiku)
Granularity	Full skills	Atomic "instincts"
Confidence	None	0.3-0.9 weighted
Evolution	Direct to skill	Instincts -> cluster -> skill/command/agent
Sharing	None	Export/import instincts

The Instinct Model

An instinct is a small learned behavior:

---
id: prefer-functional-style
trigger: "when writing new functions"
confidence: 0.7
domain: "code-style"
source: "session-observation"
scope: project
project_id: "a1b2c3d4e5f6"
project_name: "my-react-app"
---

# Prefer Functional Style

## Action
Use functional patterns over classes when appropriate.

## Evidence
- Observed 5 instances of functional pattern preference
- User corrected class-based approach to functional on 2025-01-15

Properties:

• Atomic -- one trigger, one action
• Confidence-weighted -- 0.3 = tentative, 0.9 = near certain
• Domain-tagged -- code-style, testing, git, debugging, workflow, etc.
• Evidence-backed -- tracks what observations created it
• Scope-aware -- project (default) or global

How It Works

Session Activity (in a git repo)
      |
      | Hooks capture prompts + tool use (100% reliable)
      | + detect project context (git remote / repo path)
      v
+---------------------------------------------+
|  projects/<project-hash>/observations.jsonl  |
|   (prompts, tool calls, outcomes, project)   |
+---------------------------------------------+
      |
      | Observer agent reads (background, Haiku)
      v
+---------------------------------------------+
|          PATTERN DETECTION                   |
|   * User corrections -> instinct             |
|   * Error resolutions -> instinct            |
|   * Repeated workflows -> instinct           |
|   * Scope decision: project or global?       |
+---------------------------------------------+
      |
      | Creates/updates
      v
+---------------------------------------------+
|  projects/<project-hash>/instincts/personal/ |
|   * prefer-functional.yaml (0.7) [project]   |
|   * use-react-hooks.yaml (0.9) [project]     |
+---------------------------------------------+
|  instincts/personal/  (GLOBAL)               |
|   * always-validate-input.yaml (0.85) [global]|
|   * grep-before-edit.yaml (0.6) [global]     |
+---------------------------------------------+
      |
      | /evolve clusters + /promote
      v
+---------------------------------------------+
|  projects/<hash>/evolved/ (project-scoped)   |
|  evolved/ (global)                           |
|   * commands/new-feature.md                  |
|   * skills/testing-workflow.md               |
|   * agents/refactor-specialist.md            |
+---------------------------------------------+

Project Detection

The system automatically detects your current project:

1. CLAUDE_PROJECT_DIR env var (highest priority)
2. git remote get-url origin -- hashed to create a portable project ID (same repo on different machines gets the same ID)
3. git rev-parse --show-toplevel -- fallback using repo path (machine-specific)
4. Global fallback -- if no project is detected, instincts go to global scope

Each project gets a 12-character hash ID (e.g., a1b2c3d4e5f6). A registry file at ~/.claude/homunculus/projects.json maps IDs to human-readable names.

Quick Start

1. Enable Observation Hooks

Add to your ~/.claude/settings.json.

If installed as a plugin (recommended):

{
  "hooks": {
    "PreToolUse": [{
      "matcher": "*",
      "hooks": [{
        "type": "command",
        "command": "${CLAUDE_PLUGIN_ROOT}/skills/continuous-learning-v2/hooks/observe.sh"
      }]
    }],
    "PostToolUse": [{
      "matcher": "*",
      "hooks": [{
        "type": "command",
        "command": "${CLAUDE_PLUGIN_ROOT}/skills/continuous-learning-v2/hooks/observe.sh"
      }]
    }]
  }
}

If installed manually to ~/.claude/skills:

{
  "hooks": {
    "PreToolUse": [{
      "matcher": "*",
      "hooks": [{
        "type": "command",
        "command": "~/.claude/skills/continuous-learning-v2/hooks/observe.sh"
      }]
    }],
    "PostToolUse": [{
      "matcher": "*",
      "hooks": [{
        "type": "command",
        "command": "~/.claude/skills/continuous-learning-v2/hooks/observe.sh"
      }]
    }]
  }
}

2. Initialize Directory Structure

The system creates directories automatically on first use, but you can also create them manually:

# Global directories
mkdir -p ~/.claude/homunculus/{instincts/{personal,inherited},evolved/{agents,skills,commands},projects}

# Project directories are auto-created when the hook first runs in a git repo

3. Use the Instinct Commands

/instinct-status     # Show learned instincts (project + global)
/evolve              # Cluster related instincts into skills/commands
/instinct-export     # Export instincts to file
/instinct-import     # Import instincts from others
/promote             # Promote project instincts to global scope
/projects            # List all known projects and their instinct counts

Commands

Command	Description
/instinct-status	Show all instincts (project-scoped + global) with confidence
/evolve	Cluster related instincts into skills/commands, suggest promotions
/instinct-export	Export instincts (filterable by scope/domain)
/instinct-import <file>	Import instincts with scope control
/promote [id]	Promote project instincts to global scope
/projects	List all known projects and their instinct counts

Configuration

Edit config.json to control the background observer:

{
  "version": "2.1",
  "observer": {
    "enabled": false,
    "run_interval_minutes": 5,
    "min_observations_to_analyze": 20
  }
}

Key	Default	Description
observer.enabled	false	Enable the background observer agent
observer.run_interval_minutes	5	How often the observer analyzes observations
observer.min_observations_to_analyze	20	Minimum observations before analysis runs

Other behavior (observation capture, instinct thresholds, project scoping, promotion criteria) is configured via code defaults in instinct-cli.py and observe.sh.

File Structure

~/.claude/homunculus/
+-- identity.json           # Your profile, technical level
+-- projects.json           # Registry: project hash -> name/path/remote
+-- observations.jsonl      # Global observations (fallback)
+-- instincts/
|   +-- personal/           # Global auto-learned instincts
|   +-- inherited/          # Global imported instincts
+-- evolved/
|   +-- agents/             # Global generated agents
|   +-- skills/             # Global generated skills
|   +-- commands/           # Global generated commands
+-- projects/
    +-- a1b2c3d4e5f6/       # Project hash (from git remote URL)
    |   +-- project.json    # Per-project metadata mirror (id/name/root/remote)
    |   +-- observations.jsonl
    |   +-- observations.archive/
    |   +-- instincts/
    |   |   +-- personal/   # Project-specific auto-learned
    |   |   +-- inherited/  # Project-specific imported
    |   +-- evolved/
    |       +-- skills/
    |       +-- commands/
    |       +-- agents/
    +-- f6e5d4c3b2a1/       # Another project
        +-- ...

Scope Decision Guide

Pattern Type	Scope	Examples
Language/framework conventions	project	"Use React hooks", "Follow Django REST patterns"
File structure preferences	project	"Tests in __tests__/", "Components in src/components/"
Code style	project	"Use functional style", "Prefer dataclasses"
Error handling strategies	project	"Use Result type for errors"
Security practices	global	"Validate user input", "Sanitize SQL"
General best practices	global	"Write tests first", "Always handle errors"
Tool workflow preferences	global	"Grep before Edit", "Read before Write"
Git practices	global	"Conventional commits", "Small focused commits"

Instinct Promotion (Project -> Global)

When the same instinct appears in multiple projects with high confidence, it's a candidate for promotion to global scope.

Auto-promotion criteria:

• Same instinct ID in 2+ projects
• Average confidence >= 0.8

How to promote:

# Promote a specific instinct
python3 instinct-cli.py promote prefer-explicit-errors

# Auto-promote all qualifying instincts
python3 instinct-cli.py promote

# Preview without changes
python3 instinct-cli.py promote --dry-run

The /evolve command also suggests promotion candidates.

Confidence Scoring

Confidence evolves over time:

Score	Meaning	Behavior
0.3	Tentative	Suggested but not enforced
0.5	Moderate	Applied when relevant
0.7	Strong	Auto-approved for application
0.9	Near-certain	Core behavior

Confidence increases when:

• Pattern is repeatedly observed
• User doesn't correct the suggested behavior
• Similar instincts from other sources agree

Confidence decreases when:

• User explicitly corrects the behavior
• Pattern isn't observed for extended periods
• Contradicting evidence appears

Why Hooks vs Skills for Observation?

"v1 relied on skills to observe. Skills are probabilistic -- they fire ~50-80% of the time based on Claude's judgment."

Hooks fire 100% of the time, deterministically. This means:

• Every tool call is observed
• No patterns are missed
• Learning is comprehensive

Backward Compatibility

v2.1 is fully compatible with v2.0 and v1:

• Existing global instincts in ~/.claude/homunculus/instincts/ still work as global instincts
• Existing ~/.claude/skills/learned/ skills from v1 still work
• Stop hook still runs (but now also feeds into v2)
• Gradual migration: run both in parallel

Privacy

• Observations stay local on your machine
• Project-scoped instincts are isolated per project
• Only instincts (patterns) can be exported — not raw observations
• No actual code or conversation content is shared
• You control what gets exported and promoted

Related

• ECC-Tools GitHub App - Generate instincts from repo history
• Homunculus - Community project that inspired the v2 instinct-based architecture (atomic observations, confidence scoring, instinct evolution pipeline)
• The Longform Guide - Continuous learning section

---

Instinct-based learning: teaching Claude your patterns, one project at a time.