Back to skill

Security audit

Knowledge Base with Faiss and Bailian (embedding and rerank)

Security checks for vulnerabilities and agentic risk

Overview

The skill is coherent, but needs Review because crafted knowledge-base files could cause unintended local files to be indexed and sent to the disclosed external embedding service.

Install only where KB directories are writable by trusted users, and avoid indexing sensitive or regulated files unless sending their text to Alibaba DashScope is approved. Before production use, reject symlinks and enforce resolved-path containment for all consumed files, pin dependencies, and add explicit warnings or confirmations for external processing and delete operations.

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (1)

T09 · Insecure Skill Coding Practices

Warning
Location
scripts/bailian_faiss_kb.py:291
Finding

Symlink-Based Arbitrary File Read and External Disclosure

Content
View full analysis
Path: expected_parent = kb_dir(kb_root, kb_name) doc_dir = doc_dir.resolve() if doc_dir.parent != expected_parent.resolve(): raise KBError(f"Document directory must be directly under {expected_parent}") return doc_dir ``` ```python def read_markdown_file(path: Path) -> str: return normalize_text(path.read_text(encoding="utf-8", errors="ignore")) ``` ```python for chunk_path in sorted(chunks_dir.glob("chunk-*.md")): chunk_id = parse_chunk_id(chunk_path) text = read_markdown_file(chunk_path) ``` ```python for question_path in sorted(t2q_dir.glob("*.md")): chunk_id, q_id = parse_t2q_name(question_path) if chunk_id not in chunk_lookup: raise KBError(f"T2Q file {question_path.name} references missing chunk {chunk_id}") text = collapse_text(question_path.read_text(encoding="utf-8", errors="ignore")) ``` ```python def embed_texts(texts: list[str], batch_size: int = 10) -> list[list[float]]: if not texts: return [] requests = load_requests() api_key = load_api_key() outputs: list[list[float]] = [] for start in range(0, len(texts), batch_size): batch = texts[start : start + batch_size] payload = { "model": EMBEDDING_MODEL, "input": batch, "dimensions": EMBEDDING_DIMENSIONS, "encoding_format": "float", } response = requests.post( EMBEDDING_URL, headers={"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"}, json=payload, timeout=120, ) ``` ### Technical Analysis The implementation resolves and validates the ...[truncated 3399 chars]
Remediation
View remediation
Vulnerability Patterns
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
  • Behavioral ASTexec() Call, eval() Call, Dynamic Import
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
Findings (16)

Undeclared Tool Scope

Medium
Category
MCP Least Privilege
Confidence
94% confidence
Finding

The skill describes capabilities that read environment variables, read and write local files under a configurable root, and make outbound HTTPS requests, but it does not declare any explicit tool scope such as permissions or allowed-tools. That creates a policy/visibility gap: an orchestrator or reviewer cannot reliably constrain or audit what the skill is allowed to access, and the configurable --root-dir makes filesystem reach broader than the default path suggests.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

The document is entirely prescriptive in Chinese and defines mandatory writing rules such as '必须' and '不要' for generated content, but it does not state that the skill is region-specific or that users may opt into this language requirement. Under the policy, forcing a specific language without opt-in is a natural-language locale violation.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

This markdown file instructs users to delete an index entry and then remove the corresponding document directory, but it does not include any warning that the action is destructive or irreversible. Under the markdown-specific SQP-2 criteria, descriptions of behaviors that can affect user data or system integrity should disclose that risk.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
88% confidence
Finding

The runtime notes are written entirely in Chinese, including section headings and operational instructions, with no indication that the user can choose another language or that the document is intentionally limited to a Chinese-only regional context. Under the policy, forcing a specific language without user opt-in is a natural-language policy violation.

Content

No source excerpt is available for this finding.

External Transmission

Medium
Category
Data Exfiltration
Confidence
96% confidence
Finding

The embedding request transmits raw text batches to an external API over the network. Because these texts are sourced from local document chunks and T2Q content, this can leak confidential knowledge-base contents outside the local environment, which is especially significant for enterprise or regulated data.

Content

Scanner excerpt · scripts/bailian_faiss_kb.py (reported line 602)May include surrounding context.

python
"dimensions": EMBEDDING_DIMENSIONS,
            "encoding_format": "float",
        }
        response = requests.post(
            EMBEDDING_URL,
            headers={"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"},
            json=payload,

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding

This skill sends document contents and queries to external Bailian embedding and reranking endpoints as part of normal operation. In a local knowledge-base tool, that creates a real confidentiality and privacy risk if indexed documents contain secrets, regulated data, or proprietary content, especially because the code provides no explicit consent gate, redaction step, or user-facing warning at the transmission point.

Content

No source excerpt is available for this finding.

External Transmission

Medium
Category
Data Exfiltration
Confidence
95% confidence
Finding

The rerank request sends the user query plus candidate document passages to an external reranking service. That exposes potentially sensitive searches and retrieved document snippets, which may disclose private or proprietary information beyond the local KB boundary.

Content

Scanner excerpt · scripts/bailian_faiss_kb.py (reported line 660)May include surrounding context.

python
"top_n": min(top_n, len(documents[:200])),
        "instruct": "Given a user query, rank the most relevant passages for retrieval.",
    }
    response = requests.post(
        RERANK_URL,
        headers={"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"},
        json=payload,

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
88% confidence
Finding

The delete command removes a document from stored vector records and rewrites KB artifacts; when no records remain, it also unlinks index files. Although deletion is part of the command's purpose, the code provides no confirmation prompt or explicit irreversible-operation warning before modifying persisted state.

Content

No source excerpt is available for this finding.

Dynamic import via __import__()

Medium
Category
Dangerous Code Execution
Confidence
75% confidence
Finding

Dynamic import() can load arbitrary modules at runtime, bypassing static analysis and potentially importing malicious code.

Content

Scanner excerpt · scripts/bailian_faiss_kb.py (reported line 1236)May include surrounding context.

python
def module_available(name: str) -> bool:
    try:
        __import__(name)
        return True
    except Exception:
        return False

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
84% confidence
Finding

The entire instruction file is written only in Chinese and does not indicate that language selection is optional or limited to a justified region-specific context. Per the policy rule, forcing a specific language without user opt-in can be a natural-language policy violation.

Content

No source excerpt is available for this finding.

Unpinned Dependencies

Low
Category
Supply Chain
Confidence
95% confidence
Finding

The dependency is specified with a lower bound only, which allows future unreviewed versions to be installed and prevents reproducible builds. This increases supply-chain risk because security posture and behavior can change over time without explicit approval.

Content

Scanner excerpt · requirements.txt (reported line 1)May include surrounding context.

text
faiss-cpu>=1.8.0
jieba>=0.42.1
numpy>=1.26.0
requests>=2.31.0

Unpinned Dependencies

Low
Category
Supply Chain
Confidence
95% confidence
Finding

The jieba package is not pinned to a specific version, so installations may resolve to different releases over time. That weakens reproducibility and can introduce unintended vulnerable or incompatible versions into the environment.

Content

Scanner excerpt · requirements.txt (reported line 2)May include surrounding context.

text
faiss-cpu>=1.8.0
jieba>=0.42.1
numpy>=1.26.0
requests>=2.31.0

Unpinned Dependencies

Low
Category
Supply Chain
Confidence
98% confidence
Finding

Using numpy>=1.26.0 without an exact pin means the installed version is not deterministic and may include releases with security or stability issues. In a knowledge-base skill that processes local data, this is mainly a supply-chain and reproducibility concern rather than an immediate exploit path from this file alone.

Content

Scanner excerpt · requirements.txt (reported line 3)May include surrounding context.

text
faiss-cpu>=1.8.0
jieba>=0.42.1
numpy>=1.26.0
requests>=2.31.0

Unverifiable Dependency: numpy has 16 known advisory(ies) (CVE-2014-1859 (Numpy arbitrary file write via symlink attack); CVE-2021-41495 (NumPy NULL Pointer Dereference); CVE-2021-33430 (NumPy Buffer Overflow (Disputed)) +13 more), but the manifest does not pin a version, so it is unknown whether the installed release is affected

Low
Category
Supply Chain
Confidence
90% confidence
Finding

Numpy has known advisories, and because the manifest does not pin a version, it is impossible to verify from this file whether deployment will use a fixed or affected release. This creates uncertainty in the supply chain and can leave the environment exposed if resolution selects a vulnerable build.

Content

No source excerpt is available for this finding.

Unpinned Dependencies

Low
Category
Supply Chain
Confidence
98% confidence
Finding

The requests package is version-ranged rather than pinned, which can silently introduce changed behavior or vulnerable releases during installation. Because this skill uses networked components, an unpinned HTTP client slightly raises operational risk if a bad upstream version is pulled.

Content

Scanner excerpt · requirements.txt (reported line 4)May include surrounding context.

text
faiss-cpu>=1.8.0
jieba>=0.42.1
numpy>=1.26.0
requests>=2.31.0

Unverifiable Dependency: requests has 16 known advisory(ies) (CVE-2014-1830 (Exposure of Sensitive Information to an Unauthorized Actor in Requests); CVE-2024-47081 (Requests vulnerable to .netrc credentials leak via malicious URLs); CVE-2024-35195 (Requests `Session` object does not verify requests after making first request wi) +13 more), but the manifest does not pin a version, so it is unknown whether the installed release is affected

Low
Category
Supply Chain
Confidence
93% confidence
Finding

Requests has multiple known advisories, and the absence of an exact version pin makes it unverifiable whether installs will be safe. Given this skill's likely interaction with remote services, pulling an affected requests release could expose credentials, TLS/session behavior, or other network-security properties.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.