Belief Assessor

• Outcome: Review
• Latest audit: Sep 16, 2026 · 9:17 AM UTC
• Version: 2.0.0

Overview

This skill is a coherent claim-checking tool, but it automatically pushes user claims and evidence into searches and LLM prompts with weak privacy and injection safeguards.

Review this before installing if you may use it on private, legal, medical, financial, or business-sensitive claims. It can be useful for structured claim review, but users should avoid sending confidential text to search or LLM providers and should treat its confidence scores as advisory, especially when evidence could contain hostile instructions or spoofed source URLs.

A.I.G

Vulnerability Patterns

• Insecure Skill Coding Practices: Finds exploitable flaws such as hardcoded secrets or command injection
• Skill Instruction Hijacking: Alters the agent's session goals or safety constraints when the skill loads
• Agent Memory Poisoning: Writes attacker-controlled rules into memory that affect later sessions
• Remote Payload Retrieval and Execution: Fetches external code whose behavior can change after review
• Embedded Malicious Code: Ships malicious scripts inside the skill and executes them locally

Findings (2)

T09 · Insecure Skill Coding Practices

Error

• Location: scripts/assess.py:424

Finding

Indirect Prompt Injection Through Untrusted Claim and Evidence Content

Content

View full analysis

Vulnerability Details

File Location: scripts/assess.py:424-437 and scripts/assess.py:653-666
Vulnerability Type: Indirect prompt injection caused by unsafe interpolation of untrusted content
Risk Level: High

Vulnerable Code

python

prompt = f"""You are a fact verification assistant. Based on the following [Claim] and [New Evidence], answer 6 judgments. Output only a JSON object, no other text.

Claim: {claim}
New Evidence: {evidence[:1500]}

Judgment criteria:
1. direct_support: Does the evidence directly support the claim?
2. new_info: Does the evidence provide new information not previously mentioned?
3. logical_consistent: Is the evidence logically consistent with previously known information?
4. direct_refute: Does the evidence explicitly refute the claim?
5. limitation: Does the evidence point out limitations or exceptions?
6. error_outdated: Does the evidence reveal that the claim contains errors or outdated info?

Output format:
{{"direct_support": true/false, "new_info": true/false, "logical_consistent": true/false, "direct_refute": true/false, "limitation": true/false, "error_outdated": false}}"""

The same vulnerable construction is exposed by the AI-friendly interface:

python

return f"""You are a fact verification assistant. Based on the following [Claim] and [New Evidence], answer 6 judgments. Output only a JSON object, no other text.

Claim: {claim}
New Evidence: {evidence[:1500] if evidence else "(No evidence provided)"}

Judgment criteria:
1. direct_support: Does the evidence directly support the claim?
2. new_info: Does the evidence provide new information not previously mentioned?
3. logical_consistent: Is the evidence logically consistent with previously known information?
4. direct_refute: Does the evidence explicitly refute the claim?
5. limitation: Does the evidence point out limitations or exceptions?
6. error_outdated: Does the evidence reveal that the claim contains errors or outdated info?

...[truncated 2166 chars]

Remediation

View remediation

Remediation Suggestions

1. Place the claim and evidence in clearly delimited, structured data fields rather than concatenating them directly into the instruction text.
2. Add an explicit higher-priority instruction stating that all content inside claim and evidence fields is untrusted data and that instructions found there must never be followed.
3. Use schema-constrained generation where supported, requiring exactly the six expected boolean properties and rejecting additional fields or text.
4. Validate LLM judgments independently against quoted evidence spans. Do not permit the model to mark a feature as true without identifying the supporting or refuting passage.
5. Consider a two-stage design in which one component sanitizes and extracts factual passages while a separate model evaluates only those passages.
6. Add adversarial tests covering instruction injection in claims, search snippets, page titles, URLs, and evidence bodies.
7. For high-stakes use, require human review or corroboration from multiple independently retrieved sources before reporting a claim as verified.

T09 · Insecure Skill Coding Practices

Warning

• Location: scripts/assess.py:393

Finding

Trusted Source Spoofing Through Substring-Based Domain Matching

Content

View full analysis

Vulnerability Details

File Location: scripts/assess.py:393-407
Vulnerability Type: Improper hostname validation and provenance spoofing
Risk Level: Medium

Vulnerable Code

python

def _domain_score(self, domain: str) -> float:
    domain = domain.lower()
    for pattern, score in [
        ("gov", 0.9), ("edu", 0.9), ("who.int", 0.9),
        ("pubmed.ncbi", 0.9), ("nature.com", 0.9), ("science.org", 0.9),
        ("reuters.com", 0.7), ("bbc.com", 0.7), ("apnews.com", 0.7),
        ("nytimes.com", 0.7), ("theguardian.com", 0.7),
        ("arxiv.org", 0.6), ("wikipedia.org", 0.5),
        ("twitter.com", 0.3), ("x.com", 0.3), ("reddit.com", 0.3),
    ]:
        if pattern in domain:
            return score
    if domain.endswith(".gov") or domain.endswith(".edu"):
        return 0.9
    if domain.endswith(".org"):
        return 0.6
    return 0.5

Technical Analysis

The source-reliability function treats the presence of a trusted string anywhere in a hostname as proof that the source belongs to the corresponding organization. The expression if pattern in domain does not enforce hostname-label or registrable-domain boundaries.

For example, attacker-controlled hosts such as reuters.com.attacker.example, notwho.int.example, or gov-attacker.example can receive elevated reliability scores. Generic patterns such as gov and edu are particularly broad and can occur anywhere in an unrelated hostname.

The surrounding URL extraction also uses a regular expression rather than a dedicated URL parser:

python

domains = re.findall(r'https?://([^\s/]+)', evidence)

This does not robustly normalize or validate user information, ports, internationalized names, or malformed host representations before trust scoring.

The inflated source score contributes 40% of quality_factor. That value affects raw confidence and the provenance constraint, allowing fabricated evidence to appear more trustworthy than it is.

...[truncated 1098 chars]

Remediation

View remediation

Remediation Suggestions

1. Parse URLs with urllib.parse.urlsplit() and score only the normalized hostname property.
2. Remove user information and ports through parser-based handling, reject malformed URLs, and normalize internationalized hostnames consistently.
3. Match trusted sources using exact hostname boundaries:
   • host == trusted_domain
   • host.endswith("." + trusted_domain) when verified subdomains are intended to be trusted.
4. Remove unrestricted patterns such as "gov" and "edu". If government or educational top-level domains are trusted, evaluate normalized public-suffix information rather than arbitrary substring occurrence.
5. Use a maintained public-suffix implementation to identify registrable domains and prevent suffix or subdomain confusion.
6. Maintain an explicit allowlist with documented trust levels instead of relying on keyword occurrence.
7. Add regression tests for deceptive hosts including reuters.com.attacker.example, fake-who.int.example, government.example, URL user-information tricks, ports, and mixed-case hostnames.
8. Treat domain reputation as only one weak signal and require source identity verification or corroboration before assigning high confidence.

SkillSpector

Vulnerability Patterns

• Prompt Injection: Instruction Override, Hidden Instructions, Exfiltration Commands
• Data Exfiltration: External Transmission, Env Variable Harvesting, File System Enumeration
• Trigger Abuse: Overly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
• Privilege Escalation: Excessive Permissions, Sudo/Root Execution, Credential Access
• Supply Chain: Unpinned Dependencies, External Script Fetching, Obfuscated Code

Findings (7)

Vague Triggers

Medium

• Category: Not specified by scanner
• Confidence: 95% confidence

Finding

The invocation guidance is overly broad: phrases like "is this true?" or "can I trust this?" can appear in many normal conversations and may cause the skill to activate in situations where the user did not intend a structured trust-assessment workflow. Because the skill then encourages evidence gathering and belief classification, this can lead to unnecessary handling of user content and overuse of external capabilities.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium

• Category: Not specified by scanner
• Confidence: 94% confidence

Finding

The skill explicitly says to "Always Run This First" and "MUST follow this workflow automatically," which removes meaningful gating and can force execution whenever loosely triggered. Automatic execution is risky because the workflow includes external evidence search and structured judgment generation without an explicit consent or relevance check.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium

• Category: Not specified by scanner
• Confidence: 97% confidence

Finding

The workflow mandates online evidence searches using external tools and instructs the agent to search for details about the user's claim, but it does not warn that claim contents may be transmitted to third-party services. This creates a privacy and data-governance risk, especially when claims contain sensitive personal, financial, legal, medical, or confidential organizational information.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium

• Category: Not specified by scanner
• Confidence: 85% confidence

Finding

The keyword-based reliability scoring explicitly includes Chinese-language government and media terms, which bakes locale-specific assumptions into the skill's behavior. There is no user choice, opt-in, or documented justification for this language/locale bias, so it constitutes a natural-language locale policy concern.

Content

No source excerpt is available for this finding.

Ssd 1

Medium

• Category: Not specified by scanner
• Confidence: 97% confidence

Finding

The code interpolates untrusted claim and evidence text directly into an instruction prompt for an LLM, allowing adversarial evidence to include prompt-injection content such as instructions to ignore prior directions or emit attacker-chosen JSON. Because the tool treats the LLM output as structured feature labels that drive confidence and state transitions, injected content can manipulate assessment outcomes and degrade the integrity of downstream reasoning.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium

• Category: Not specified by scanner
• Confidence: 91% confidence

Finding

The skill sends the full claim and up to 1500 characters of evidence to an LLM via llm_func, but the public API and helper interfaces do not disclose that potentially sensitive user-supplied content will be transmitted to an external model provider. In an agent setting, claims and evidence may contain private, regulated, or proprietary data, so silent exfiltration to a remote service creates a real privacy and compliance risk.

Content

No source excerpt is available for this finding.

Ssd 1

Medium

• Category: Not specified by scanner
• Confidence: 98% confidence

Finding

The helper returns a ready-to-send prompt that embeds attacker-controlled claim and evidence text verbatim for another AI agent to answer. This increases risk because it exports prompt-injection payloads across component boundaries: any downstream agent consuming that prompt may follow malicious instructions embedded in the evidence, causing corrupted judgments or broader agent misbehavior depending on that agent's privileges.

Content

No source excerpt is available for this finding.