Back to skill

Security audit

Agent Eval

Security checks for vulnerabilities and agentic risk

Overview

The skill is a coherent agent-evaluation rubric, but its automatic workflow reads broad task history and writes persistent agent-memory records without clear user control.

Install only if you are comfortable giving the evaluating agent access to broad task-history memory and allowing it to write persistent evaluation and pattern records. Prefer using sanitized task summaries, explicit approval before SOUL.md or model/config inspection, and human review before persistent pattern or PAT updates.

Vulnerability Patterns
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (1)

T05 · Unauthorized Access and Privilege Escalation

Warning
Location
SKILL.md:129
Finding

Overbroad Access to Task History and Persistent Agent State

Content
View full analysis
.md` ### 每周日 00:00 — CEO 周报 1. 汇总各 agent 周平均分 2. 识别分数下降趋势 → 触发调查 3. 识别高分稳定 agent → 确认无需干预 4. 提炼本周 top-3 失败点 → 写入 patterns.md 5. 将整体评分趋势发给老板 ### 触发优化的条件 - 连续3天某 agent 低于 70 分 → 自动 SOUL.md 检查 - 某个 eval 项连续5次失败 → 写入 PAT 记录 - 周均分下降 >10% → 触发 agent 模型/配置审查 ``` ### Technical Analysis The evaluation workflow instructs the agent to read all task records for a given day rather than a narrowly scoped and sanitized evaluation dataset. Daily task records may contain unrelated conversations, personal information, credentials, confidential business data, or other sensitive material that is unnecessary for calculating evaluation scores. The workflow also persists derived information to `memory/evolution/.md`, `patterns.md`, and PAT records, and it can initiate inspection of `SOUL.md` and agent model or configuration data. The document does not define authorization checks, per-agent isolation, sensitive-data redaction, retention limits, permitted fields, or read-only constraints for configuration inspection. Consequently, an evaluator granted these instructions could cross least-privilege boundaries by accessing unrelated historical records and sensitive agent state. Untrusted content contained in task records could also be summarized or copied into persistent files, increasing its lifetime and making it available to later agents or sessions. ### Attack Path 1. A user, external document, or prior agent task introduces sensitive or attacker-controlled content into `memory/YYYY-MM-DD.md`. 2. The scheduled daily evaluation process reads every task record for that date, including content unrelated to the e ...[truncated 1729 chars]
Remediation
View remediation
Vulnerability Patterns
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
Findings (1)

Natural-Language Policy Violations

Medium
Category
Not specified by scanner
Confidence
91% confidence
Finding

This markdown skill file presents all instructions, evaluation criteria, and operational guidance exclusively in Chinese. Under the natural-language policy rules, forcing a specific language without user opt-in is a locale/language policy violation unless the constraint is explicitly justified, which is not stated here.

Content

No source excerpt is available for this finding.