Security audit

ab-testing

Security checks for vulnerabilities and agentic risk

Overview

The skill is mostly consistent with A/B testing, but it reaches into shared tenant analytics data and describes ongoing behavior-log collection and executive notifications without enough scoping or user control.

Review this skill before installing in any environment with real tenants, production analytics, behavior logs, or executive messaging. Prefer running it only where database credentials are least-privilege, tenant access is enforced outside the skill, stored experiment data has a retention policy, and automated notifications require review.

SkillSpector

Vulnerability Patterns

Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands

Findings (6)

Lp3

Medium

Category: MCP Least Privilege
Confidence: 86% confidence
Finding: The skill declares only `read` and `exec`, but the document also describes capabilities that imply broader access such as reading memory logs, environment/runtime constraints, and file-based storage interactions. This kind of permission/capability mismatch is dangerous because it obscures the real trust boundary, making reviewers and users underestimate what data the skill may access or how it operates.

Intent-Code Divergence

Medium

Confidence: 82% confidence
Finding: The document's core-function statement is inconsistent with the later design, which describes reading test data, collecting behavior logs, writing reports, cron-driven analysis, and CEO notifications. Contradictory documentation can conceal sensitive behavior and lead operators to invoke the skill under false assumptions, increasing the chance of unauthorized data handling or unsafe automation.

Description-Behavior Mismatch

Medium

Confidence: 87% confidence
Finding: The skill is presented as a local A/B testing framework, but the compare path reaches into tenant-scoped production analytics data from a shared PostgreSQL database. That expands the trust boundary and allows a caller with access to this script to enumerate or analyze real tenant content performance, which is a capability materially different from simple experiment bookkeeping.

Context-Inappropriate Capability

Medium

Confidence: 90% confidence
Finding: The script reads shared database configuration and opens pooled connections for analytics queries without any visible permission check or justification tied to the caller. In a skill context, this creates unnecessary access to shared infrastructure and can expose cross-system data if the skill is triggered in broader contexts than intended.

Missing User Warnings

Medium

Confidence: 91% confidence
Finding: The skill describes reading user behavior logs and writing test configurations/results into `memory/ab-tests/`, but does not clearly warn users about this data access and persistence. This is risky because behavioral data may contain sensitive or identifying information, and silent persistence can create privacy, retention, and integrity issues if users do not expect their data to be stored and reprocessed.

Missing User Warnings

Medium

Confidence: 94% confidence
Finding: The workflow includes automatic daily collection of user behavior data and an automated CEO notification path, yet the document lacks a privacy and integrity warning for this ongoing surveillance and escalation mechanism. In context, this is more dangerous because the skill is user-invocable, scheduled, and tied to business decision-making, so unreviewed automation could expose sensitive metrics, propagate incorrect conclusions, or send data to senior stakeholders without consent or validation.

Static analysis

No suspicious patterns detected.