Security audit

Rag Accuracy Optimizer

Security checks across malware telemetry and agentic risk

Overview

This is a coherent RAG optimization skill, but users should be careful because several examples can send prompts, document chunks, or evaluation data to cloud AI providers and can log raw queries locally.

Install only if you want a documentation-and-script skill for improving RAG systems. Before running examples, use a controlled environment, review dependencies, avoid real customer, healthcare, financial, legal, or proprietary data with remote providers unless approved, and redact or disable raw query/context logging for sensitive deployments.

SkillSpector

By NVIDIA

Vulnerability Patterns

Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access

Findings (15)

Context-Inappropriate Capability

Medium

Confidence: 90% confidence
Finding: The reference example includes live external LLM API calls in a document primarily about chunking patterns, which expands the trust boundary from local text processing to third-party data transmission. Because the code sends chunk text to an external model without strong justification, opt-in controls, or discussion of privacy constraints, users may copy it into production and unintentionally expose sensitive corpus content.

Context-Inappropriate Capability

Medium

Confidence: 95% confidence
Finding: The example directly reads API keys from environment variables and forwards arbitrary prompts to multiple third-party LLM providers without any data-classification gate, consent check, redaction step, or tenant policy enforcement. In a RAG setting, prompts often contain proprietary documents, customer data, or regulated information, so this pattern can cause unauthorized external disclosure and expand the attack surface across several vendors.

Context-Inappropriate Capability

Medium

Confidence: 97% confidence
Finding: The monitoring component persistently stores raw queries and session identifiers to local JSONL files, creating a durable record of potentially sensitive user inputs without minimization, retention limits, access controls, or masking. If logs are accessed by unauthorized users, copied to backups, or ingested into analytics systems, they can expose personal data, proprietary prompts, credentials, or confidential business information.

Intent-Code Divergence

Medium

Confidence: 95% confidence
Finding: The PostgreSQL example claims parameterized execution but still interpolates `alpha` and `top_k` directly into the SQL text via an f-string. If either value is derived from untrusted input, this can enable SQL manipulation or at minimum unsafe query construction and denial-of-service style abuse through oversized limits. In a reference skill, readers may copy this pattern into production, which increases practical risk.

Missing User Warnings

Medium

Confidence: 93% confidence
Finding: The skill recommends logging raw queries, retrieved chunks, and generated answers in production without any minimization, redaction, retention, consent, or access-control guidance. In real deployments this can capture sensitive user prompts, proprietary corpus excerpts, or regulated data, creating privacy, compliance, and secondary data-exposure risk.

Missing User Warnings

Medium

Confidence: 88% confidence
Finding: The file includes code that sends document chunks and queries to external OpenAI embedding/chat APIs, but it does not warn users that potentially sensitive corpus data may leave their environment. In a RAG optimization skill, users are likely to test with proprietary business, insurance, finance, or healthcare documents, so omission of a privacy/data-handling warning creates a real risk of unintended data disclosure.

Missing User Warnings

Low

Confidence: 76% confidence
Finding: The Markdown recommends GraphRAG CLI indexing and query commands over user documents without noting privacy or governance implications. While less direct than embedded code, these commands can still cause users to process sensitive corpora and generate derived artifacts without understanding where data is stored or how it is handled.

Missing User Warnings

Medium

Confidence: 97% confidence
Finding: The code transmits up to 2000 characters of each chunk to an external API via `client.chat.completions.create(...)` with no user-facing warning, consent step, or data classification check. In a RAG context, chunks often contain proprietary, regulated, or personal data, so this example can normalize unsafe exfiltration of corpus contents to a third party.

Missing User Warnings

Medium

Confidence: 96% confidence
Finding: The sample code writes raw user queries to disk with no warning, notice, or opt-in, which creates a privacy and compliance risk even if the storage is intended for observability. Users and integrators may assume ephemeral processing, while the implementation silently creates persistent records that may include sensitive content.

Missing User Warnings

Medium

Confidence: 95% confidence
Finding: The fallback chain transmits prompts to Google, OpenAI, and Anthropic without any accompanying privacy warning or transmission control, so adopters may unknowingly send sensitive user or retrieved document content to external processors. Multi-provider fallback increases exposure because the same request may traverse several vendors during failures, complicating consent, residency, and contractual compliance.

Missing User Warnings

Medium

Confidence: 87% confidence
Finding: The query rewriting example sends user queries and optional context to an external LLM API without warning that the data leaves the local environment. In RAG systems, query context can contain sensitive business, customer, financial, or healthcare information, so undocumented transmission creates a real privacy and compliance risk. The skill context makes this more concerning because it explicitly targets domain-specific deployments such as insurance, finance, and healthcare.

Missing User Warnings

Medium

Confidence: 93% confidence
Finding: The reranking and compression patterns transmit document text to third-party APIs, which can expose internal knowledge-base contents, customer data, or regulated records. Because these examples are part of a RAG optimization skill and include production-oriented guidance, users may apply them to sensitive corpora without realizing the privacy impact. That makes the omission more dangerous than in a purely toy example.

Missing User Warnings

Medium

Confidence: 92% confidence
Finding: The example code sends raw question, context, and answer data to an external LLM provider for evaluation, but the documentation does not warn that this may transmit proprietary, regulated, or sensitive content off-platform. In a RAG setting, contexts often contain internal documents or customer data, so readers may copy this pattern into production and unintentionally create a data exposure path.

Missing User Warnings

Medium

Confidence: 93% confidence
Finding: When OpenAI or Cohere models are selected, the script sends query and dataset text to third-party APIs without an explicit privacy warning or consent gate. In a RAG accuracy workflow, benchmark datasets may contain proprietary, regulated, or customer data, so silent transmission outside the local environment can cause confidentiality and compliance issues.

Missing User Warnings

Medium

Confidence: 93% confidence
Finding: This script passes user-supplied test data into RAGAS metrics via `evaluate(...)`, and several RAGAS metrics are commonly backed by external LLM/API calls. The input includes questions, answers, retrieved contexts, and ground truth, so potentially sensitive evaluation corpora can be transmitted off-host without any explicit warning, consent gate, redaction, or local-only option, creating a real data exposure risk.

VirusTotal

64/64 vendors flagged this skill as clean.

View on VirusTotal

Static analysis

Detected: suspicious.prompt_injection_instructions

Prompt-injection style instruction pattern detected.

Warn

Code: suspicious.prompt_injection_instructions
Location: references/testing-frameworks.md:295

Prompt-injection style instruction pattern detected.

Warn

Code: suspicious.prompt_injection_instructions
Location: SKILL.md:355