Semantic Circuit Breaker

Protect downstream services by monitoring semantic content quality and triggering circuit breaks based on semantic drift, inconsistency, factual errors, or t...

Security Scan

The skill's description and algorithm match its stated purpose, but the runtime instructions are under-specified and grant the agent broad discretion to choose embedding/verification services and data sources, which is ambiguous and could lead to unexpected access or data exfiltration.

Purpose & Capability

Name/description (semantic circuit breaker) align with the SKILL.md: it describes semantic-distance triggers and a three-state breaker. However, the instructions rely on functions like embed() and verify_against_kb() without specifying where embeddings, the LLM, or the knowledge base come from—operational dependencies that are necessary to implement the described capability are not declared.

Instruction Scope

The SKILL.md is high-level and leaves implementation choices (which embedding provider, which KB, how to fetch/verify facts, thresholds, how many responses N, where baseline comes from) up to the agent. Vague, open-ended guidance can cause the agent to access arbitrary external services or data sources or to request credentials at runtime.

Install Mechanism

Instruction-only skill with no install spec or code files — minimal surface area and nothing is written to disk by the skill itself.

Credentials

The skill declares no required environment variables or credentials, but its examples require embedding, LLM calls, and KB verification. In practice these will need API keys/credentials; because they are not declared, the agent might attempt to use whatever credentials it already has access to or reach out to external endpoints — an under-specified credential model.

Persistence & Privilege

No always:true, no install-time persistence, and no modification of other skills or system-wide config is indicated. The skill runs only when invoked (user-invocable) and does not request elevated persistence.

What to consider before installing

This skill conceptually fits its name, but the runtime instructions are deliberately vague about where to get embeddings, which LLM or KB to use, and what thresholds to apply. Before installing or enabling this skill: 1) require the author to specify which embedding/LLM/KB endpoints it will call and what credentials it needs; 2) only grant it access to vetted providers or a local embedding service, not broad platform credentials; 3) review and lock down any KB or external endpoints to avoid unintentional data leaks; 4) ask for concrete defaults (N, thresholds, baseline source) or implement these policies yourself; and 5) test in a restricted environment first. If the author supplies concrete integration targets and declared env vars, reassess — the current ambiguity is why I mark it as suspicious.

语义熔断器：基于内容语义而非错误率的服务保护机制。

何时使用

• API返回"正确格式但错误语义"的响应（如AI幻觉）
• 需要保护下游服务免受语义漂移影响
• LLM API的质量保障层

核心认知

1. 传统熔断看状态码，语义熔断看内容

传统Circuit Breaker基于HTTP状态码和超时率。但当LLM API返回200但内容是幻觉、偏见、或语义偏移时，传统熔断完全无效。

语义熔断触发条件：

• 语义漂移：连续N个响应的语义embedding偏离历史基线超过阈值
• 一致性断裂：对同一问题的多次回答语义差异过大
• 事实性崩塌：响应中可验证事实的错误率超过阈值
• 毒性飙升：响应中的有害内容评分突增

2. 三态模型（扩展传统熔断）

• Closed（正常）：语义质量在基线范围内，请求正常通过
• Open（熔断）：语义质量严重偏离，请求被拦截并返回降级响应
• Half-Open（探测）：定期发送探测请求，评估语义质量是否恢复

区别：转换条件不是"错误率"而是"语义距离"。

3. 语义距离计算

def semantic_distance(responses, baseline):
    """计算响应集合与基线的语义距离"""
    embeddings = embed(responses)  # 句向量
    baseline_emb = embed(baseline)
    
    # 余弦距离
    cosine_dist = 1 - cosine_similarity(embeddings.mean(axis=0), baseline_emb.mean(axis=0))
    
    # 方差（一致性）
    variance = embeddings.std(axis=0).mean()
    
    # 事实性（与知识库比对）
    factuality = verify_against_kb(responses)
    
    return 0.4 * cosine_dist + 0.3 * variance + 0.3 * (1 - factuality)

碰撞来源

• circuit-breaker-pattern × semantic-diff-engine × llm-output-validator
• semantic-cache-layer（语义计算）× adaptive-circuit-breaker（自适应熔断）