Meta Knowledge Base

AI-powered knowledge base builder that automatically captures, organizes, and retrieves information. Learns from conversations, documents, and interactions t...

Security Scan

The skill mostly matches a local knowledge-base prototype, but the README claims heavier auto-capture, web scraping, and production embedding behavior while the shipped code contains simplified placeholders and the docs ask you to install large embedding libraries that the code does not use — this mismatch and the auto-capture guidance warrant caution.

Purpose & Capability

Name and description (self-building KB, RAG, semantic search) align with the provided code (KnowledgeBase, vector store, graph, add/search/ask). However SKILL.md advertises capabilities (real embedding models, web scraping, file-watch, email parsing, continuous background learning) that are only stubbed or simplified in the code: embeddings are generated by a local hash/random function, add_from_url does not fetch remote content (stores a placeholder string), and file-watch/web-scrape/email parsing are described but not implemented or incomplete. The README also instructs pip installing heavy libraries (faiss-cpu, sentence-transformers) while the code does not import or use them — a disproportionate dependency request relative to the shipped code.

Instruction Scope

SKILL.md encourages 'auto-capture' from conversations, documents, web pages and shows an integration snippet hooking into after_message to call kb.add(...) — that implies automatic ingestion of user conversations and files. The code writes all captured content to disk under ~/.meta_knowledge/<name> and will index any content passed to add/add_from_file. While this behavior is coherent for a KB, it means sensitive messages/files could be stored locally automatically. The instructions are permissive (hooks and 'implicit learning') without explicit guidance about filtering, consent, or redaction. This raises privacy risk if installed without restricting what is auto-captured.

Install Mechanism

There is no formal install spec in the package; SKILL.md recommends running 'pip install numpy faiss-cpu sentence-transformers'. Those are heavy, platform-sensitive packages (faiss-cpu in particular can be problematic on some OSes). The included code does not import or use sentence-transformers or faiss; embeddings are produced locally with a hash/random fallback. Requiring these dependencies in docs but not using them is an inconsistency and may lead users to install unnecessary large packages.

Credentials

The skill requests no environment variables, no external credentials, and no config paths beyond writing into a user directory (~/.meta_knowledge/<name>). That is proportionate to a local knowledge-base. Note: while no network credentials are requested, the documentation suggests web scraping and message hooks; if you enable such hooks in a larger system, the skill will have access to whatever conversation or file data the host supplies — so restrict what is passed to it.

Persistence & Privilege

The skill does not request 'always: true' and does not modify other skills/configs. It persists data to a local path in the user's home directory and manages its own files, which is expected for a KB. Autonomous invocation is permitted by default (not flagged here) but should be considered together with the auto-capture guidance.

What to consider before installing

This appears to be a local KB prototype rather than a production-ready ingestion agent. Before installing or enabling auto-capture: 1) Review and test the Python file in a sandbox — the code writes everything passed to add()/add_from_file() into ~/.meta_knowledge/<name>/*.json. 2) Do not enable automatic hooks (after_message, file-watch, email parsing) without auditing what data will be sent; they can cause sensitive messages/files to be stored. 3) You do not need to blindly run the pip install line: the code as shipped uses a local dummy embedding generator; installing faiss-cpu and sentence-transformers is optional and heavy — only install them if you plan to replace the stubbed embedding function with a real model and understand platform implications. 4) If you intend to use remote fetching (add_from_url) or real embeddings, inspect and modify those methods to ensure safe network behavior and to add rate-limiting, timeouts, and explicit consent. 5) If you want a production deployment, request or implement explicit filters, redaction, and access controls so the KB does not capture secrets automatically.

Self-building knowledge management system that learns and grows automatically.

Features

1. Auto-Capture

• Conversation Learning: Extract key information from chats
• Document Parsing: Extract from PDFs, docs, emails
• Web Scraping: Learn from visited pages
• File Watch: Monitor folders for new content

2. Knowledge Organization

• Auto-Tagging: Automatic topic categorization
• Entity Extraction: People, companies, concepts
• Relationship Mapping: Connect related ideas
• Version History: Track knowledge evolution

3. Semantic Search

• Vector Embeddings: Semantic similarity search
• Hybrid Search: Combine keyword + semantic
• Filtering: Filter by date, tags, source
• Ranking: Relevance-based results

4. Intelligent Q&A

• RAG Pipeline: Retrieve + Generate answers
• Context-Aware: Understand conversation context
• Citing Sources: Reference original knowledge
• Confidence Scoring: Show answer confidence

5. Continuous Learning

• User Feedback: Learn from corrections
• Implicit Learning: Learn from interactions
• Knowledge Updates: Keep information fresh
• Gap Identification: Find missing knowledge

Installation

pip install numpy faiss-cpu sentence-transformers

Usage

Initialize Knowledge Base

from meta_knowledge import KnowledgeBase

kb = KnowledgeBase(
    name="my_knowledge",
    embedding_model="paraphrase-multilingual-MiniLM-L12-v2"
)

Add Knowledge

# From text
kb.add(
    content="Python is a high-level programming language...",
    tags=["programming", "python"],
    metadata={"source": "user", "date": "2026-03-22"}
)

# From document
kb.add_from_file("document.pdf", tags=["research"])

# From URL
kb.add_from_url("https://example.com/article", tags=["news"])

Search

# Semantic search
results = kb.search(
    query="What is machine learning?",
    top_k=5
)

for r in results:
    print(f"{r.score:.2f} | {r.content[:100]}...")

Q&A

# Ask questions
answer = kb.ask(
    question="What do I know about AI?",
    include_sources=True
)

print(answer['answer'])
print("Sources:", answer['sources'])

Knowledge Graph

# Get entity relationships
graph = kb.get_knowledge_graph()

# Find related concepts
related = kb.find_related("Python", depth=2)

API Reference

Adding Knowledge

Method	Description
add(content, ...)	Add single piece of knowledge
add_batch(contents)	Add multiple items
add_from_file(path)	Parse and add file
add_from_url(url)	Fetch and add web content
add_from_email(email)	Parse email content

Searching

Method	Description
search(query, top_k)	Semantic search
hybrid_search(query, ...)	Keyword + semantic
filter_search(query, filters)	Search with filters
find_similar(content)	Find similar items

Q&A

Method	Description
ask(question, ...)	Get answer with RAG
get_context(question)	Get relevant context
generate_summary(topic)	Generate topic summary

Management

Method	Description
get_knowledge_graph()	Get entity relationships
list_tags()	List all tags
export(format)	Export knowledge
import_(data)	Import knowledge

Architecture

┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│   Sources   │────▶│  Ingestion  │────▶│   Storage    │
│ - Chat      │     │ - Parser    │     │ - Vector DB  │
│ - Docs      │     │ - Embedder  │     │ - Graph DB   │
│ - Web       │     │ - Indexer   │     │ - Document   │
└─────────────┘     └─────────────┘     └─────────────┘
                                           │
                    ┌──────────────────────┘
                    ▼
┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│   Query     │────▶│   Retrieve  │────▶│   Generate  │
│ - Search    │     │ - Vector    │     │ - LLM       │
│ - Ask       │     │ - Graph     │     │ - Cite      │
└─────────────┘     └─────────────┘     └─────────────┘

Embedding Models

Model	Dimensions	Languages	Use Case
paraphrase-multilingual-MiniLM-L12-v2	384	50+	General
bge-small-zh-v1.5	512	Chinese	Chinese
text-embedding-ada-002	1536	EN	Production

Use Cases

• Personal Assistant: Remember everything
• Team Wiki: Shared knowledge base
• Customer Support: Q&A automation
• Research: Paper search & summarization
• Codebase: Documentation search

Best Practices

1. Regular Updates: Keep knowledge fresh
2. Quality over Quantity: Clean data matters
3. Use Tags: Organize for better retrieval
4. User Feedback: Improve with corrections
5. Backup: Export regularly

Integration

With OpenClaw

# Auto-capture from conversations
@hookimpl
def after_message(message, response):
    kb.add(
        content=f"User asked about: {extract_topics(message)}",
        tags=["conversation", extract_topics(message)]
    )

With Skills

# Use knowledge in skills
def my_skill(query):
    context = kb.search(query, top_k=3)
    return generate_response(query, context)

Future Capabilities

• Multi-modal knowledge (images, audio)
• Real-time sync across devices
• Collaborative knowledge base
• Automatic knowledge validation