Pattern Miner

Automatically detects repeated code and command patterns in Python/Shell, generating reusable Jinja2 templates and shell automation scripts via CLI.

Security Scan

The skill's code and instructions are broadly consistent with its stated purpose (discovering repeated code/command patterns and generating templates/scripts), but it will read potentially sensitive local files (shell histories, session logs) and generate runnable artifacts—so review how and where it runs before enabling automated scans.

Purpose & Capability

Name/description (pattern mining, template/script generation) match the included code and CLI: analyzers for code and shell history, template and script generators, and Node/Python components. The skill legitimately needs file access to workspace files, history, and configured sources. Minor note: SKILL.md emphasizes ML techniques (KMeans, LOF, TF-IDF) while some shown Python analyzer code uses AST/heuristic sliding-window detection; ML implementations may reside in other files (TS/python modules) not fully shown.

Instruction Scope

Runtime instructions explicitly tell the agent to read session logs (~/.openclaw/sessions), decision logs (~/.openclaw/decisions), workspace files, and standard shell history files (~/.bash_history, ~/.zsh_history). This matches the skill purpose, but these sources can contain sensitive data (API keys, secrets, private conversation contents). The SKILL.md also suggests scheduled/crontab runs and autoScan — enabling those will create persistent, recurring reads of sensitive local data.

Install Mechanism

Registry contains package.json and many source files but no platform install-spec; SKILL.md instructs manual installation (npm install / npm run build and pip install for Python deps). Requested packages are common (commander, fs-extra, scikit-learn, pandas, tree-sitter). No external arbitrary download URLs were seen in provided files, but installation will pull many npm/pip packages—standard but should be done from trusted registries and inspected if you have elevated threat concerns.

Credentials

The skill does not request environment variables, secrets, or config paths in the metadata. It does, however, read local files (history, sessions, SQLite DBs) that can contain credentials or other sensitive strings; these reads are proportionate to its stated function (mining user workflow patterns) but increase exposure risk if the agent is run without restrictions.

Persistence & Privilege

always:false (no forced global inclusion). The SKILL.md recommends optional scheduled runs (crontab or autoScan). Autonomous invocation is allowed by default for skills on-platform (normal), but enabling autoScan/crontab gives the skill recurring filesystem access — consider this when granting the skill runtime privileges.

Assessment

This skill appears to do what it says: read local workspace files and shell history to find repeated code/command patterns and generate templates/scripts. Before installing or enabling automatic scans: 1) Review the full source (especially TS/python files not fully reviewed) for any network calls or unexpected endpoints. 2) Run initial analyses on a small, non-sensitive directory to observe outputs. 3) Do NOT enable autoScan or add the crontab until you’re comfortable with what data is collected and where outputs are stored (by default ~/.pattern-miner and configurable output dirs). 4) Be cautious because shell history and session logs may contain secrets; consider sanitizing sources or running the skill in a constrained/sandboxed account. 5) Install dependencies from official registries and inspect package-lock/package.json for any unusual packages. If you want, I can scan the remaining truncated files (TS and python modules) for network I/O or other suspicious behavior — that would raise confidence to high.

pattern-miner Skill

Intelligent pattern recognition and actionable insights from multi-source data.

Description

The pattern-miner skill discovers hidden patterns in your workflow data (conversations, decisions, tasks) using machine learning techniques. It identifies recurring themes, associations, and anomalies, then generates actionable insights to improve productivity and decision-making.

Installation

cd ~/.openclaw/workspace/skills/pattern-miner
npm install
npm run build

Python Dependencies

pip install numpy scikit-learn pandas tree-sitter

Usage

CLI Commands

# Run pattern mining
pattern-miner mine

# Incremental mining (only new data)
pattern-miner mine --incremental

# List discovered patterns
pattern-miner list
pattern-miner list --type cluster
pattern-miner list --verbose

# Analyze specific patterns/insights
pattern-miner analyze
pattern-miner analyze --pattern <id>
pattern-miner analyze --insight <id>
pattern-miner analyze --category optimization

# Apply insights
pattern-miner apply --confirm
pattern-miner apply --insight <id> --confirm
pattern-miner apply --category automation --dry-run

# Show statistics
pattern-miner stats

# Export patterns
pattern-miner export --format json --output patterns.json
pattern-miner export --format csv --output patterns.csv

# Configuration
pattern-miner config --show
pattern-miner config --init

Node.js API

import { PatternMiner } from '@openclaw/skill-pattern-miner';

const miner = new PatternMiner({
  minConfidence: 0.7,
  minFrequency: 5,
  analysisTypes: ['cluster', 'association', 'anomaly']
});

await miner.initialize();

// Run mining
const results = await miner.mine();
console.log(`Found ${results.summary.totalPatterns} patterns`);
console.log(`Generated ${results.summary.totalInsights} insights`);

// List patterns
const patterns = await miner.listPatterns();
const clusterPatterns = await miner.listPatterns('cluster', 10);

// List insights
const insights = await miner.listInsights(undefined, true); // pending only

// Get stats
const stats = await miner.getStats();

// Apply insight
await miner.applyInsight('insight_123');

// Export
const json = await miner.exportPatterns('json');
const csv = await miner.exportPatterns('csv');

Core Features

Multi-Source Data Collection

Automatically collects data from:

• Conversations: Session logs from context-preserver
• Decisions: Decision records from decision-recorder
• Tasks: Task files in workspace (JSON, Markdown)
• Files: Any file patterns you configure

Intelligent Pattern Recognition

Three analysis types:

1. Clustering (cluster)

   • Groups similar items using KMeans
   • Identifies recurring themes and topics
   • TF-IDF vectorization for text similarity

2. Association Rules (association)

   • Finds items that frequently occur together
   • Calculates confidence and support metrics
   • Discovers hidden relationships

3. Anomaly Detection (anomaly)

   • Identifies outliers using Local Outlier Factor
   • Flags unusual patterns for review
   • Helps catch edge cases and issues

Pattern Scoring System

Each pattern is scored on:

• Confidence: How reliable the pattern is (0-1)
• Frequency: How often the pattern appears
• Importance: Composite score based on:
  • Pattern confidence
  • Frequency normalized to max
  • Item priority metadata

Actionable Insights

Generated insights include:

• Title: Clear description of the finding
• Description: Context and metrics
• Action: Specific recommendation
• Priority: high/medium/low
• Expected Impact: Estimated value (0-1)
• Category: optimization/automation/risk

Configuration

Default config is stored at ~/.pattern-miner/config.json:

{
  "dataDir": "~/.openclaw/workspace",
  "patternDir": "~/.pattern-miner",
  "minConfidence": 0.6,
  "minFrequency": 3,
  "analysisTypes": ["cluster", "association", "anomaly"],
  "sources": [
    {
      "type": "conversation",
      "name": "conversations",
      "path": "~/.openclaw/sessions",
      "pattern": "**/*.json"
    }
  ],
  "autoScan": false,
  "scanInterval": 60,
  "maxPatterns": 1000,
  "retentionDays": 30
}

Data Storage

Patterns and insights are stored in ~/.pattern-miner/:

• patterns.json - Discovered patterns
• insights.json - Generated insights
• config.json - Configuration

Integration

With context-preserver

The skill automatically reads conversation logs if context-preserver is installed:

{
  "sources": [
    {
      "type": "conversation",
      "name": "sessions",
      "path": "~/.openclaw/sessions",
      "pattern": "**/*.json"
    }
  ]
}

With decision-recorder

Integrates with decision logs:

{
  "sources": [
    {
      "type": "decision",
      "name": "decisions",
      "path": "~/.openclaw/decisions",
      "pattern": "**/*.json"
    }
  ]
}

Scheduled Mining

For automatic periodic scanning, add to your crontab:

# Run pattern mining every hour
0 * * * * cd ~/.openclaw/workspace/skills/pattern-miner && pattern-miner mine --incremental

Or enable auto-scan in config:

{
  "autoScan": true,
  "scanInterval": 60
}

Output Examples

Pattern Output

{
  "id": "cluster_0_1710234567",
  "type": "cluster",
  "items": ["...", "..."],
  "confidence": 0.85,
  "frequency": 12,
  "importance": 0.78,
  "metadata": { "centroid": [...] },
  "createdAt": "2024-03-12T09:00:00Z",
  "source": "clustering"
}

Insight Output

{
  "id": "insight_cluster_0_1710234567",
  "patternId": "cluster_0_1710234567",
  "title": "Recurring Pattern: Code review feedback...",
  "description": "Found 12 similar items forming a pattern",
  "action": "Review and standardize this pattern",
  "priority": "high",
  "expectedImpact": 0.7,
  "category": "optimization"
}

Troubleshooting

Python not found

Ensure Python 3.8+ is installed and in PATH:

python3 --version

No patterns found

• Check that data sources are configured correctly
• Ensure there's enough data (minFrequency defaults to 3)
• Try running with --verbose to see collection details

Low confidence scores

• Increase data volume
• Adjust minConfidence in config
• Check data quality and consistency

Technical Details

Algorithms

• Clustering: KMeans with TF-IDF features
• Association: Apriori-style rule mining
• Anomaly: Local Outlier Factor (LOF)

Performance

• Incremental scanning for large datasets
• Configurable pattern retention (default 30 days)
• Max pattern limit (default 1000)

License

MIT