Back to skill

Security audit

Review Llm Artifacts

Security checks for vulnerabilities and agentic risk

Overview

This skill is a repository review helper that reads project files and writes a local JSON report, with its behavior generally disclosed and aligned to its purpose.

Before installing, expect the skill to read repository source files, optionally scan the whole project when requested with --all, use local CLI tools, and write a .beagle/llm-artifacts-review.json report into the working tree. Review that generated file before committing it.

Vulnerability Patterns
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
Vulnerability Patterns
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
Findings (4)

Vague Triggers

Medium
Category
Not specified by scanner
Confidence
89% confidence
Finding

The manifest description says the skill triggers on 'dead code sweeps,' 'test-quality passes,' and 'when the user asks to scan the whole repo.' These are broad natural-language conditions rather than a specific invocation list, which increases the chance of unintended activation during ordinary repository review conversations.

Content

No source excerpt is available for this finding.

Description-Behavior Mismatch

Medium
Category
Not specified by scanner
Confidence
94% confidence
Finding

The skill is presented as a review/detection tool, but it also persists a JSON artifact inside the repository workspace. Writing files changes the working tree and can have unintended side effects in CI, pre-commit workflows, or downstream automation that trusts repo contents, so this exceeds a strictly read-only review capability.

Content

No source excerpt is available for this finding.

Intent-Code Divergence

Medium
Category
Not specified by scanner
Confidence
93% confidence
Finding

The prose claims a historical staleness check against a previous report, but the actual shell snippet reads .beagle/llm-artifacts-review.json after Step 5 has written the current report and then compares that stored head to the current one. That logic does not implement the documented 'previous report exists' behavior and will not meaningfully detect report staleness as described.

Content

No source excerpt is available for this finding.

Context-Inappropriate Capability

Low
Category
Not specified by scanner
Confidence
72% confidence
Finding

The declared purpose is artifact review, but the implementation depends on multiple local command executions for repository inspection, directory creation, JSON parsing, and integrity checks. While some local file inspection is expected, requiring command execution and environment tooling is a broader capability than the manifest explicitly communicates.

Content

No source excerpt is available for this finding.

Static analysis

No suspicious patterns detected.