Semantic Vector Store

提供文本向量化、持久化存储及基于余弦相似度的语义搜索，支持增量索引和多后端适配。

Security Scan

The skill generally matches a local vector-store purpose but has multiple inconsistencies and small security/design risks (missing dependency/install info, docs/registry mismatch, and use of pickle for persisted data) that warrant caution before installing.

Purpose & Capability

Name/description describe a vector store and the included code implements a local SQLite+FAISS-backed vector store with embedding support — this is coherent. However, the SKILL.md claims multi-backend support (Pinecone, Weaviate) while the code focuses on local SQLite+FAISS; the registry metadata declares no required env vars while SKILL.md documents several configuration env vars (SEMANTIC_VECTOR_DB_PATH, EMBEDDING_MODEL, etc.). No homepage or source is provided (unknown origin).

Instruction Scope

Runtime instructions and examples are limited to expected operations (init, embed, add, search). The implementation writes a database file under the user's home (~/.config/cortexgraph/semantic_vectors.db) and reads/writes FAISS index files and .ids.pkl files. The code uses pickle to serialize/deserialize embeddings and ID mappings; untrusted or tampered DB/index files could lead to unsafe pickle deserialization. SKILL.md does not warn about this or about trusting DB files.

Install Mechanism

There is no install spec in the registry (instruction-only), but the shipped Python file depends on external packages (sentence-transformers, faiss, numpy) with no declared installation steps or dependency manifest. SKILL.md shows a 'clawhub install' command but there is no explicit dependency installation instructions or pinned sources, which may confuse users and lead to manual, ad-hoc installs.

Credentials

The registry lists no required env vars or credentials, yet SKILL.md documents several configurable env vars (DB path, EMBEDDING_MODEL, VECTOR_DIMENSION, SIMILARITY_THRESHOLD). No credentials are requested (no network API keys), which is proportional to a local vector store. The mismatch between registry requirements and documented env vars is an inconsistency to be aware of.

Persistence & Privilege

The skill persists data to disk in a user config directory and saves FAISS index files and pickle id-maps — this is expected for a vector store. The skill is not marked 'always: true' and does not claim to alter other skills' configs. The notable risk is the local persistence format (pickle), which can execute code when loading if files are maliciously modified.

What to consider before installing

This skill appears to implement a local vector store (SQLite + optional FAISS) and mostly does what it claims, but exercise caution: 1) There is no verified source/homepage — only run it if you trust the publisher or after reviewing the code. 2) Dependencies (sentence-transformers, faiss, numpy) are not declared in the registry; install them in a controlled virtual environment. 3) The implementation uses pickle for persisted data and ID maps — do not point the skill at untrusted DB or index files (pickle deserialization can run arbitrary code). 4) SKILL.md advertises multi-backend support and environment variables, but the code primarily supports a local SQLite+FAISS path — verify the connectors you need are actually implemented. 5) If you decide to install, run it inside an isolated environment (container/VM) until you confirm behavior; consider reviewing the rest of the code (the truncated portion) to ensure there are no network callbacks or hidden endpoints.

语义向量库插件，为 OpenClaw 星型记忆架构提供向量存储与语义搜索能力。

🎯 功能

• 向量化记忆：将记忆文本转换为语义向量（嵌入）
• 向量存储：持久化存储向量，支持高效检索
• 语义搜索：基于余弦相似度的向量搜索
• 增量索引：支持动态添加新记忆，无需全量重建
• 可插拔后端：支持 SQLite + FAISS、Pinecone、Weaviate 等后端

📦 安装

# 从 ClawHub 安装
clawhub install semantic-vector-store

# 或从本地目录安装
clawhub install ./skills/semantic-vector-store

⚙️ 配置

环境变量：

• SEMANTIC_VECTOR_DB_PATH：向量数据库路径（默认：~/.config/cortexgraph/semantic_vectors.db）
• EMBEDDING_MODEL：嵌入模型名称（默认：all-MiniLM-L6-v2）
• VECTOR_DIMENSION：向量维度（默认：384）
• SIMILARITY_THRESHOLD：相似度阈值（默认：0.5）

🚀 快速开始

from semantic_vector_store import SemanticVectorStore

# 初始化
store = SemanticVectorStore()

# 添加记忆向量
mem_id = "mem_001"
text = "今天学习了 Python 异步编程"
vector = store.embed(text)
store.add_vector(mem_id, vector, metadata={"source": "memory.md"})

# 语义搜索
query = "编程学习"
results = store.search(query, top_k=5)
for result in results:
    print(f"记忆 ID: {result['memory_id']}, 相似度: {result['score']:.3f}")

🔌 适配器接口

SemanticVectorAdapter

提供星型架构的标准适配器接口：

from integration.adapter.semantic_vector_adapter import SemanticVectorAdapter

adapter = SemanticVectorAdapter()

# 获取健康状态
health = adapter.health_check()
print(f"健康状态: {health['status']}")

# 向量化记忆
vectors = adapter.embed_memory("今天完成了重要工作")
print(f"生成向量维度: {len(vectors)}")

# 语义搜索
results = adapter.search("工作", limit=10)

主要方法

方法	说明
embed(text: str) -> List[float]	将文本转换为向量
add_vector(memory_id: str, vector: List[float], metadata: dict)	添加向量到存储
search(query: str, top_k: int = 10) -> List[dict]	语义搜索
get_stats() -> dict	获取统计信息
health_check() -> dict	健康检查

🏗️ 架构设计

核心组件

1. 嵌入器（Embedder）

   • 支持多种嵌入模型（Sentence Transformers、OpenAI、本地模型）
   • 模型缓存与热加载

2. 向量存储（VectorStore）

   • SQLite + FAISS 混合存储
   • 支持增量索引与批量导入
   • 自动向量归一化

3. 查询引擎（QueryEngine）

   • 余弦相似度计算
   • 混合搜索（语义 + 关键词）
   • 结果排序与去重

数据模型

CREATE TABLE semantic_vectors (
    memory_id TEXT PRIMARY KEY,
    vector BLOB,           -- FAISS 索引 ID 或原始向量
    metadata TEXT,         -- JSON 元数据
    created_at TIMESTAMP,
    updated_at TIMESTAMP
);

🔄 集成点

与 Memory Sync Enhanced 集成

语义向量库通过适配器与 MSE 集成：

# 在 MSE 中使用语义向量库
from integration.adapter.semantic_vector_adapter import SemanticVectorAdapter

class MemorySyncEnhanced:
    def __init__(self):
        self.vector_adapter = SemanticVectorAdapter()
    
    def semantic_search(self, query: str):
        return self.vector_adapter.search(query)

与 Unified Memory 集成

统一记忆系统可同时查询语义向量和共现图：

from unified_memory import UnifiedMemory

memory = UnifiedMemory()
results = memory.search("查询", use_semantic=True, use_cooccurrence=True)

📊 性能指标

指标	目标值
向量化延迟	< 100ms（短文本）
搜索延迟	< 50ms（10K 向量）
存储容量	> 1M 向量
准确率	> 0.85（MRR）

🔧 维护

数据库维护

# 重建索引
python3 -m semantic_vector_store.reindex

# 备份向量
python3 -m semantic_vector_store.backup --output vectors.bin

# 查看统计
python3 -m semantic_vector_store.stats

监控指标

• 向量存储使用率
• 查询延迟（P50, P95, P99）
• 缓存命中率
• 嵌入模型加载状态

📝 版本历史

• v0.1.0（当前）：MVP 版本，基础向量存储与搜索
• v0.2.0（计划）：支持 FAISS 索引，性能优化
• v0.3.0（计划）：多模型支持，混合搜索

🤝 贡献

欢迎提交 Issue 和 Pull Request。请遵循 OpenClaw 技能开发规范。

📄 许可证

MIT