Security audit

REFINE: The Self-Evolving Agent

Security checks for vulnerabilities and agentic risk

Overview

The skill is locally scoped and mostly transparent, but it persists diagnostic text and turns stored error messages into future system-prompt patches in a way users should review carefully.

Install only if you are comfortable with a local diagnostics file that may contain feedback and exception-message text. Do not pass secrets, raw prompts, customer data, or attacker-controlled error text into this skill, and do not automatically prepend generated patch_body content to a system prompt without human review and sanitization.

A.I.G

Vulnerability Patterns

Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally

Findings (3)

T01 · Skill Instruction Hijacking

Error

Location: main.py:397
Finding: Persistent System-Prompt Injection Through Stored Exception Messages
Content: View full analysis
Remediation: View remediation

T09 · Insecure Skill Coding Practices

Error

Location: main.py:78
Finding: Arbitrary Sensitive Text Is Persisted and Logged Without Content Redaction
Content: View full analysis
str: """ Truncate a string to _MAX_TEXT characters before storage. 截断字符串至最大长度，防止大量内容写入磁盘。 """ s = str(text).strip() return (s[:_MAX_TEXT] + "…") if len(s) > _MAX_TEXT else s ``` Feedback is passed through truncation only: ```python record = { "id": _uid(), "timestamp": _ts(), "feedback": _safe(feedback), "context": _sanitize_context(context), } self.session["feedback"].append(record) ``` Exception messages are also only truncated and are then written to application logs: ```python first_line = str(error).split("\n")[0] record = { "id": _uid(), "timestamp": _ts(), "type": type(error).__name__, "message": _safe(first_line), "context": _sanitize_context(context), } self.session["errors"].append(record) self.memory["error_log"].append(record) self.memory["stats"]["total_errors"] += 1 log.error("Error [%s]: %s — %s", record["id"][:8], record["type"], record["message"]) return record ``` ### Technical Analysis The code-level sensitive-key filtering implemented by `_sanitize_context()` applies only to context dictionaries. The separate `_safe()` function used for feedback and exception messages performs no content redaction; it merely converts input to a string, strips surrounding whitespace, and truncates it. Consequently, credentials, API tokens, personal data, raw prompts, authorization headers, URLs containing secrets, or other confidential values remain intact if supplied as feedback or included in an exception's first line. Feedback is persisted when `close()` saves the session. Exception messages are inserted into both the session and the global error log and are also immediately sent to the configured l ...[truncated 1375 chars]
Remediation: View remediation

T09 · Insecure Skill Coding Practices

Warning

Location: main.py:330
Finding: History API Returns Full Sensitive Session Records Instead of Summaries
Content: View full analysis
list[dict]: """ Return N most recent session count summaries (no raw text). 返回最近 N 次会话统计摘要（不含原始内容）。 """ sessions = self.memory.get("sessions", []) return sessions[-limit:][::-1] ``` A complete session object contains the following fields: ```python return { "id": _uid(), "opened_at": _ts(), "closed_at": None, "summary": "", "feedback": [], "errors": [], } ``` ### Technical Analysis The method's documentation states that it returns count summaries without raw text. The implementation instead slices and returns the original session dictionaries. Those dictionaries contain complete feedback records, error messages, and context values. No projection, field filtering, copying into a summary schema, or access-control check occurs. Consumers that trust the documented contract may therefore pass the returned data to user interfaces, telemetry systems, model contexts, or external callers under the assumption that it contains only aggregate counts. The use of a negative `limit` also produces unintuitive slicing behavior, although the primary security issue is the disclosure of full records. ### Attack Path 1. The engine stores feedback, errors, and associated context in completed sessions. 2. A component invokes `get_history()` based on the documented promise that only summaries are returned. 3. The method returns complete session dictionaries. 4. The component displays, serializes, logs, or otherwise forwards the result. 5. Any sensitive information present in feedback, error messages, or context is exposed to the downstream recipient. ### Impact Assessment The issue can disclose all stored feedback, error records, and context data for the sessions selecte ...[truncated 370 chars]
Remediation: View remediation
list[dict]: if not isinstance(limit, int) or limit < 1: raise ValueError("limit must be a positive integer") limit = min(limit, 100) sessions = self.memory.get("sessions", []) return [ { "id": session.get("id"), "opened_at": session.get("opened_at"), "closed_at": session.get("closed_at"), "feedback_count": len(session.get("feedback", [])), "error_count": len(session.get("errors", [])), } for session in sessions[-limit:][::-1] ] ``` ]]>

SkillSpector

Vulnerability Patterns

Behavioral ASTexec() Call, eval() Call, Dynamic Import
MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access

Findings (2)

Undeclared Tool Scope

Medium

Category: MCP Least Privilege
Confidence: 91% confidence
Finding: The skill describes capabilities to read environment variables and write to local disk, but it does not declare an explicit tool/permission scope such as allowed-tools or permissions. That creates a mismatch between documented intent and enforceable boundaries, increasing the risk of unintended access to secrets in environment variables or broader file operations than reviewers expect.
Content: No source excerpt is available for this finding.

Dynamic attribute access via getattr()

Low

Category: Dangerous Code Execution
Confidence: 50% confidence
Finding: Dynamic getattr() with a non-literal attribute name can access arbitrary object attributes, potentially bypassing access controls.
Content: Scanner excerpt · main.py (reported line 58)May include surrounding context.
python
# ─── Logging ───────────────────────────────────────────────────────────────── logging.basicConfig( level = getattr(logging, LOG_LEVEL, logging.INFO), format = "%(asctime)s [%(levelname)s] %(name)s — %(message)s", datefmt = "%Y-%m-%dT%H:%M:%S", )

Static analysis

No suspicious patterns detected.