Semantic Consistency Auditor

• Outcome: Review
• Latest audit: May 28, 2026 · 2:19 PM UTC
• Version: 1.0.0

Overview

The skill appears to be a local clinical-note semantic scoring tool, but it is packaged with misleading academic-writing framing and weak disclosure around sensitive medical text, model downloads, and dependency risk.

Review before installing. Use only with de-identified clinical or similarly sensitive text, run it in an isolated environment, fix the syntax error, replace and pin dependencies, verify model sources, and treat any JSON or console output as sensitive because it can include the original AI-generated and gold-standard text.

SkillSpector

Vulnerability Patterns

• Data Exfiltration: External Transmission, Env Variable Harvesting, File System Enumeration
• Supply Chain: Unpinned Dependencies, External Script Fetching, Obfuscated Code
• Excessive Agency: Unrestricted Tool Access, Autonomous Decision Making, Scope Creep
• MCP Least Privilege: Underdeclared Capability, Wildcard Permission, Missing Permission Declaration
• MCP Tool Poisoning: Hidden Instructions, Unicode Deception, Parameter Description Injection

Findings (17)

Lp3

Medium

• Category: MCP Least Privilege
• Confidence: 91% confidence
• Finding: The skill documentation advertises executable paths and file-based JSON input/output, which implies file read/write capability without any declared permissions or trust boundary. This is dangerous because agents or reviewers may treat the skill as lower risk than it is, leading to unintended access to local files and output locations.

Tp4

High

• Category: MCP Tool Poisoning
• Confidence: 98% confidence
• Finding: The declared purpose says this is an academic-writing workflow aid, but the body describes a medical semantic evaluation tool that downloads external models and processes clinical-note content. This mismatch is dangerous because it can mislead users and automated policy systems into approving a skill for benign academic use when it actually handles sensitive medical text and network-enabled model execution.

Description-Behavior Mismatch

High

• Confidence: 97% confidence
• Finding: The manifest description frames the skill as an academic-writing auditor, while the content clearly targets clinical-note semantic evaluation. Mislabeling capability and domain is dangerous because it defeats informed consent, misroutes the skill into inappropriate contexts, and increases the chance that sensitive healthcare data will be processed under weaker controls.

Intent-Code Divergence

High

• Confidence: 96% confidence
• Finding: The 'When to Use' guidance reinforces an academic-writing workflow framing, but the rest of the document is for clinical semantic scoring. This is dangerous because operators may invoke the skill on the wrong data types or approve it for environments that are not authorized for medical-data handling or external model downloads.

Description-Behavior Mismatch

Medium

• Confidence: 84% confidence
• Finding: The skill performs runtime model download/load behavior that is not apparent from the stated auditing purpose, which expands the trust boundary and introduces unreviewed network and supply-chain exposure. In a workflow handling sensitive medical text, hidden dependency retrieval can cause privacy, availability, and integrity risks if external resources are compromised or blocked.

Context-Inappropriate Capability

Medium

• Confidence: 88% confidence
• Finding: Runtime downloading of the COMET model is a real supply-chain and operational risk because it pulls executable model artifacts from external infrastructure during normal use. For a tool processing medical content, this is especially concerning when the manifest does not clearly justify or disclose the need for live downloads in potentially regulated environments.

Missing User Warnings

Medium

• Confidence: 88% confidence
• Finding: The skill encourages evaluation of clinical notes and writing detailed JSON outputs but does not warn against inclusion of real patient identifiers or other sensitive health data. This is dangerous because users may process regulated medical information and persist it to disk without de-identification, retention limits, or access controls.

Missing User Warnings

Low

• Confidence: 76% confidence
• Finding: The installation and performance notes imply downloading large external models, but the documentation does not clearly disclose the network dependency and associated privacy or supply-chain considerations. This is risky because users may run the skill in restricted or sensitive environments without understanding that external services and third-party model artifacts are involved.

Unpinned Dependencies

Low

• Category: Supply Chain

Content

bert_score
comet
dataclasses
numpy

• Confidence: 93% confidence
• Finding: The dependency list is unpinned, so installs may pull different versions over time, including newly introduced vulnerable or incompatible releases. In an academic writing workflow that likely processes external models and ML packages, this weakens supply-chain integrity and makes builds non-reproducible.

Unpinned Dependencies

Low

• Category: Supply Chain

Content

bert_score
comet
dataclasses
numpy
torch

• Confidence: 93% confidence
• Finding: The unpinned comet dependency allows package resolution to drift to unexpected versions, which can introduce security issues or break behavior without review. Because this skill depends on ML/NLP tooling, uncontrolled upgrades can silently expand attack surface or pull compromised transitive dependencies.

Unpinned Dependencies

Low

• Category: Supply Chain

Content

bert_score
comet
dataclasses
numpy
torch
yaml

• Confidence: 84% confidence
• Finding: dataclasses is listed without a version pin, which contributes to non-reproducible dependency resolution. However, this package is often unnecessary on modern Python versions because dataclasses is built into Python 3.7+, so the security impact here is lower than for the heavier ML packages.

Unpinned Dependencies

Low

• Category: Supply Chain

Content

bert_score
comet
dataclasses
numpy
torch
yaml

• Confidence: 97% confidence
• Finding: An unpinned numpy dependency is dangerous because it can resolve to versions with known vulnerabilities or breaking ABI/API changes. Since numpy is a foundational package with broad transitive use, version drift can propagate risk across much of the environment.

Unpinned Dependencies

Low

• Category: Supply Chain

Content

comet
dataclasses
numpy
torch
yaml

• Confidence: 98% confidence
• Finding: Leaving torch unpinned is especially risky because PyTorch has a large attack surface, frequent security advisories, and features such as model loading that have historically enabled serious exploitation. In a workflow likely to handle ML artifacts, uncontrolled torch version selection materially increases the chance of installing a vulnerable build.

Unpinned Dependencies

Low

• Category: Supply Chain

Content

dataclasses
numpy
torch
yaml

• Confidence: 99% confidence
• Finding: The yaml entry is both unpinned and suspicious because it does not match the common PyYAML package name, increasing supply-chain risk. If a package named yaml is resolved from the index, an attacker-controlled or unintended package could be installed with arbitrary malicious code during setup or runtime.

Known Vulnerable Dependency: numpy — 10 advisory(ies): CVE-2014-1859 (Numpy arbitrary file write via symlink attack); CVE-2021-41495 (NumPy NULL Pointer Dereference); CVE-2021-33430 (NumPy Buffer Overflow (Disputed)) +7 more

Critical

• Category: Supply Chain
• Confidence: 91% confidence
• Finding: The file declares numpy without a version, and static analysis reports multiple advisories affecting some numpy releases. While the requirement alone does not prove an exploitable vulnerable version will be installed, the absence of pinning means a known-bad release could be selected depending on environment and resolver behavior.

Known Vulnerable Dependency: torch — 10 advisory(ies): CVE-2025-2953 (PyTorch susceptible to local Denial of Service); CVE-2022-45907 (PyTorch vulnerable to arbitrary code execution); CVE-2025-32434 (PyTorch: `torch.load` with `weights_only=True` leads to remote code execution) +7 more

Critical

• Category: Supply Chain
• Confidence: 96% confidence
• Finding: The requirement includes torch with no version constraints, and PyTorch has had high-impact issues including unsafe deserialization and code execution paths. In a semantic auditing skill using ML components, this is more dangerous because model loading and tensor processing are plausible operations, increasing exposure to vulnerable functionality.

Possible Typosquatting: 'yaml' resembles popular package 'pyyaml'

High

• Category: Supply Chain
• Confidence: 99% confidence
• Finding: yaml is a classic typosquatting concern because the expected Python package is typically pyyaml, not yaml. If the wrong package is fetched, the installation process can introduce attacker-controlled code, making this a severe supply-chain compromise risk rather than a simple functionality bug.

Static analysis

No suspicious patterns detected.