Back to skill

Security audit

agent-verifier

Security checks for vulnerabilities and agentic risk

Overview

The skill is a coherent pre-send message verifier, but its safety checks can incorrectly approve sensitive or manipulated outbound messages and its third-party review path is under-disclosed.

Review this before installing if you will rely on it for regulated or confidential outbound messaging. Use a trusted local or approved LLM only, avoid the hosted endpoint for sensitive drafts unless Workloft's data handling is acceptable, and do not treat a PASS as a strong confidentiality guarantee until the redlist recipient exception and LLM response validation are hardened.

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (2)

T09 · Insecure Skill Coding Practices

Error
Location
agent_verifier/verifier.py:252
Finding

Semantic Verification Can Be Bypassed Through Prompt Injection and Fail-Open Response Parsing

Content
View full analysis

Vulnerability Details

File Location: agent_verifier/verifier.py, lines 79–123 and 252–310
Vulnerability Type: Prompt injection and fail-open security validation
Risk Level: High

Vulnerable Code

python
_DEFAULT_PROMPT = """You are reviewing a draft outbound message before it goes to a human recipient.

Recipient: {recipient}
Campaign: {campaign}
Subject: {subject}
Body:
{body}

Do THREE things:

1. **Atomic-claim decomposition.** Break the body into 1–10 atomic claims
   — small units of fact or intent that could be independently true or false.
...
Return ONLY a JSON object with this exact shape (no markdown fences):

{{
  "atomic_claims": [
    {{"claim": "<verbatim or close paraphrase>", "status": "verified|unverifiable|failed", "reason": "<one sentence>"}}
  ],
  "confidential": {{"severity": "PASS|WARN|BLOCK", "reason": "<short>"}},
  "clarity":      {{"severity": "PASS|WARN|BLOCK", "reason": "<short>"}},
  "could_not_verify": "<one sentence, or empty string>",
  "needs_from_user": "<one sentence, or empty string>"
}}"""
python
prompt = self._prompt.format(
    recipient=recipient, campaign=campaign,
    subject=subject, body=body[:self._max_body],
)
try:
    raw = self._llm(prompt)
except Exception as e:
    return (CheckResult("WARN", "llm",
                        f"LLM call failed ({e!r}); defaulting to WARN"),
            *empty_extras)
text = (raw or "").strip()
if text.startswith("```"):
    text = text.split("```", 2)[1]
    if text.lower().startswith("json"):
        text = text[4:]
    text = text.rsplit("```", 1)[0].strip()
try:
    verdict = json.loads(text)
except Exception:
    return (CheckResult("WARN", "llm",
                        "LLM returned non-JSON; defaulting to WARN"),
            *empty_extras)

# Atomic claims — bias toward unverifiable when the model is silent.

...[truncated 3856 chars]
Remediation
View remediation

Remediation Suggestions

  1. Validate LLM responses against a strict schema before calculating a verdict. Require:
    • atomic_claims to be an array.
    • confidential and clarity to be objects.
    • Both holistic axes to contain a recognized PASS, WARN, or BLOCK severity.
    • All required feedback fields to have their documented types.
  2. Fail closed when required fields are missing, malformed, or have unexpected types. A malformed security verdict should produce BLOCK, or at minimum WARN, rather than PASS.
  3. Use provider-supported structured-output or JSON-schema enforcement where available.
  4. Separate trusted verifier instructions from untrusted message data using system/user message roles when supported by the LLM provider.
  5. Place untrusted fields in explicit structured delimiters and state that text inside those fields is data, not instructions. This reduces but does not eliminate prompt-injection risk.
  6. Consider applying independent deterministic checks to model output and requiring an explicit, complete attestation before permitting a semantic PASS.
  7. Add adversarial tests covering:
    • A body requesting an empty JSON response.
    • Missing confidential or clarity axes.
    • Empty objects and arrays.
    • Invalid severity values.
    • Wrong field types.
    • Attempts to place forged JSON or prompt instructions in every interpolated field.

T09 · Insecure Skill Coding Practices

Error
Location
agent_verifier/verifier.py:205
Finding

Recipient Substring Exception Allows Confidential Redlist Bypass

Content
View full analysis

Vulnerability Details

File Location: agent_verifier/verifier.py, lines 205–229
Vulnerability Type: Improper authorization based on substring matching
Risk Level: High

Vulnerable Code

python
def _redlist_check(self, subject: str, body: str,
                   recipient: str) -> CheckResult:
    if not self._redlist_terms and not self._redlist_regexes:
        return CheckResult("PASS", "redlist", "no redlist configured")
    haystack = f"{subject}\n{body}".lower()
    recip = (recipient or "").lower()
    hits: list[str] = []
    for term in self._redlist_terms:
        if term in haystack and term not in recip:
            hits.append(term)
    for rx in self._redlist_regexes:
        for m in rx.finditer(haystack):
            hit = m.group(0)
            if hit.lower() not in recip:
                hits.append(f"regex({rx.pattern}): {hit}")
    if hits:
        sample = "; ".join(sorted(set(hits))[:5])
        return CheckResult(
            "BLOCK", "redlist",
            f"Confidential term(s) leaked into outbound: {sample}",
        )
    return CheckResult("PASS", "redlist", "no redlisted terms")

Technical Analysis

The verifier suppresses a redlist match whenever the matched term occurs anywhere in the raw recipient string. This behavior assumes that textual inclusion of a confidential term proves that the recipient is the authorized related party.

Email addresses are attacker-selectable strings and do not establish authorization through substring membership. For example, if ProjectAtlas is redlisted, unrelated addresses such as projectatlas@attacker.example or user@projectatlas-attacker.example satisfy the exception.

The same weakness applies to regex matches. If the exact matched text appears in the recipient string, the regex finding is suppressed even though no trusted mailbox or domain relationship has been established.

The check does not par ...[truncated 1565 chars]

Remediation
View remediation

Remediation Suggestions

  1. Remove the recipient substring exception by default. A redlisted term should block regardless of its appearance in the recipient string unless explicit authorization exists.
  2. If related-party exceptions are required, define an explicit mapping between each redlist entry and approved normalized email addresses or domains.
  3. Parse recipient values with a standards-aware email parser and compare the normalized mailbox or domain exactly.
  4. Do not authorize recipients through local-part matches, display names, suffixes, substring containment, or lookalike domains.
  5. For domain authorization, require exact domain equality or a carefully controlled subdomain relationship; do not use unrestricted endswith() checks.
  6. Keep confidentiality terms and recipient authorization policies separate so updating one does not implicitly change the other.
  7. Add tests for:
    • Confidential terms in attacker-controlled local parts.
    • Lookalike and suffix domains.
    • Display-name injection.
    • Multiple recipient addresses.
    • Case and Unicode normalization.
    • Regex matches copied into unrelated recipient addresses.
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
Findings (9)

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

The README promotes a hosted verification endpoint for checking outbound message content but does not warn that message bodies, subjects, recipients, and related metadata may be transmitted to a third-party service. In this skill's context, the verifier is explicitly intended to inspect potentially confidential outbound communications, so omission of data-handling, retention, and privacy guidance can lead operators to expose regulated or sensitive content unintentionally.

Content

No source excerpt is available for this finding.

Undeclared Tool Scope

Medium
Category
MCP Least Privilege
Confidence
70% confidence
Finding

Without declared permissions the skill's intent is opaque and cannot be validated.

Content

No source excerpt is available for this finding.

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

The 'When to use this' section says to use the skill in 'any agent stack that sends outbound text on the firm's behalf' and lists broad categories like helpdesk replies, social posts, and notifications. For a markdown skill description, this is an expansive activation scope with no negative examples or clear limits on when the skill should not be invoked, which risks unintended use.

Content

No source excerpt is available for this finding.

Missing User Warnings

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

The verifier sends the full outbound subject/body, recipient, and campaign data to an arbitrary LLM callable for semantic review, but the code provides no built-in consent, disclosure, minimization, or trust-boundary enforcement. In this skill's context, that is particularly sensitive because the component is explicitly meant to inspect potentially confidential outbound communications, so it may exfiltrate the very secrets it is supposed to protect.

Content

No source excerpt is available for this finding.

Scope Creep

Low
Category
Excessive Agency
Confidence
70% confidence
Finding

Skill's behavior or capabilities extend beyond its stated purpose. Scope creep allows an agent to perform actions unrelated to its documented functionality, increasing the attack surface.

Content

Scanner excerpt · LICENSE (reported line 12)May include surrounding context.

text
the Software, and to permit persons to whom the Software is furnished to do so.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION

Missing User Warnings

Low
Category
Not specified by scanner
Confidence
78% confidence
Finding

When a redlist path is provided, the verifier reads a local file from disk to load confidential terms and regexes. Although this is not inherently unsafe, the code path performs file access without any nearby warning, logging, or explicit note beyond parameter naming, which may leave operators unaware of local data access behavior.

Content

No source excerpt is available for this finding.

Unverifiable Dependency: setuptools has 10 known advisory(ies) (CVE-2013-1633 (Setuptools vulnerable to Man-in-the-middle attacks); CVE-2025-47273 (setuptools has a path traversal vulnerability in PackageIndex.download that lead); CVE-2024-6345 (setuptools vulnerable to Command Injection via package URL) +7 more), but the manifest does not pin a version, so it is unknown whether the installed release is affected

Low
Category
Supply Chain
Confidence
40% confidence
Finding

Dependency has known vulnerabilities (CVEs). Using packages with unpatched security flaws exposes the environment to known exploits.

Content

No source excerpt is available for this finding.

Unverifiable Dependency: pytest has 2 known advisory(ies) (CVE-2025-71176 (pytest has vulnerable tmpdir handling); CVE-2025-71176 (pytest has vulnerable tmpdir handling)), but the manifest does not pin a version, so it is unknown whether the installed release is affected

Low
Category
Supply Chain
Confidence
40% confidence
Finding

Dependency has known vulnerabilities (CVEs). Using packages with unpatched security flaws exposes the environment to known exploits.

Content

No source excerpt is available for this finding.

Natural-Language Policy Violations

Low
Category
Not specified by scanner
Confidence
85% confidence
Finding

The test treats use of the American spelling 'organize' as a style warning via the string literal "americanism: 'organize'". This reflects a natural-language locale policy preference being applied automatically, with no indication of user choice or opt-in.

Content

No source excerpt is available for this finding.