Embeddings

Generate, store, and search vector embeddings with provider selection, chunking strategies, and similarity search optimization.

Security Scan

This is an instruction-only embeddings guide that is internally consistent with its stated purpose — it provides examples and best practices but does not itself request credentials, install code, or perform unexpected actions.

Purpose & Capability

The name/description match the content: guidance on generating, chunking, storing, and searching embeddings across many providers and storage backends. The examples show typical SDK usage for OpenAI, Cohere, Voyage, local models, Pinecone, Qdrant, pgvector, Chroma, etc., which is expected for a comprehensive embeddings handbook.

Instruction Scope

SKILL.md and the included files contain only standard, domain-appropriate instructions and code snippets (chunking, batching, normalization, vector DB queries, reranking, query expansion). There are no instructions to read unrelated system files, harvest environment variables, contact unknown endpoints, or exfiltrate data. The guidance does assume calling external provider APIs (which requires network access and credentials) and using local model libraries, which is expected for the stated functionality.

Install Mechanism

No install spec and no code files — instruction-only. This minimizes risk because nothing is written/executed by the skill itself. The docs reference common third-party libraries (OpenAI SDK, langchain, tiktoken, sentence-transformers, provider clients) but do not attempt to install them.

Credentials

The skill declares no required env vars or credentials, which is coherent for an instructions-only skill. However, the examples presuppose provider credentials and API keys (OpenAI, Pinecone, Cohere, etc.) and show inline placeholders (e.g., api_key="..."). Users will need to supply appropriate keys to actually run the snippets. This is proportionate but worth highlighting: providing provider keys gives those services access to any data you send for embedding, so credentials and data choice matter.

Persistence & Privilege

always is false and the skill does not request any persistent or elevated system privileges. It does not instruct changing other skills' configs or system-wide settings. Autonomous invocation is allowed by platform default but not uniquely privileged by this skill.

Assessment

This file bundle is a coherent, offline guide — it doesn't include executable code or request secrets itself. Before using the code snippets: (1) Understand that to call provider APIs you must supply API keys/credentials — only provide keys you trust and scope them if possible. (2) Embedding sensitive or private data will send that data to whichever provider you call (unless you run local models); consider privacy/compliance and prefer local models or on-prem storage for sensitive data. (3) The examples assume various third-party libraries and vector DBs; you will need to install and configure them separately. (4) If you let an agent run these workflows autonomously, ensure network access and credentials are limited to the intended providers and monitor API usage/costs. Overall this skill appears to be documentation/best-practices rather than an executable integration and is internally consistent.

When to Use

User wants to convert text/images to vectors, build semantic search, or integrate embeddings into applications.

Quick Reference

Topic	File
Provider comparison & selection	providers.md
Chunking strategies & code	chunking.md
Vector database patterns	storage.md
Search & retrieval tuning	search.md

Core Capabilities

1. Generate embeddings — Call provider APIs (OpenAI, Cohere, Voyage, local models)
2. Chunk content — Split documents with overlap, semantic boundaries, token limits
3. Store vectors — Insert into Pinecone, Weaviate, Qdrant, pgvector, Chroma
4. Similarity search — Query with top-k, filters, hybrid search
5. Batch processing — Handle large datasets with rate limiting and retries
6. Model comparison — Evaluate embedding quality for specific use cases

Decision Checklist

Before recommending approach, ask:

• What content type? (text, code, images, multimodal)
• Volume and update frequency?
• Latency requirements? (real-time vs batch)
• Budget constraints? (API costs vs self-hosted)
• Existing infrastructure? (cloud provider, database)

Critical Rules

• Same model everywhere — Query embeddings MUST use identical model as document embeddings
• Normalize before storage — Most similarity metrics assume unit vectors
• Chunk with overlap — 10-20% overlap prevents context loss at boundaries
• Batch API calls — Never embed one item at a time in production
• Cache embeddings — Regenerating is expensive; store with source hash
• Monitor dimensions — Higher isn't always better; 768-1536 is usually optimal

Provider Quick Selection

Need	Provider	Why
Best quality, any cost	OpenAI text-embedding-3-large	Top benchmarks
Cost-sensitive	OpenAI text-embedding-3-small	5x cheaper, 80% quality
Multilingual	Cohere embed-multilingual-v3	100+ languages
Code/technical	Voyage voyage-code-2	Optimized for code
Privacy/offline	Local (e5, bge, nomic)	No data leaves machine
Images	OpenAI CLIP, Cohere multimodal	Cross-modal search

Common Patterns

# Batch embedding with retry
def embed_batch(texts, model="text-embedding-3-small"):
    results = []
    for chunk in batched(texts, 100):  # API limit
        response = client.embeddings.create(input=chunk, model=model)
        results.extend([e.embedding for e in response.data])
    return results

# Similarity search with filter
results = index.query(
    vector=query_embedding,
    top_k=10,
    filter={"category": "technical"},
    include_metadata=True
)