Back to skill

Security audit

Alchemyst MCP Skill

Security checks for vulnerabilities and agentic risk

Overview

The skill is a legitimate external context-memory integration, but it under-scopes persistent agent instructions and proactive remote retrieval in ways users should review before installing.

Install only if you intend to use Alchemyst as an external persistent context store. Do not store secrets, credentials, regulated data, or sensitive internal conversations unless you have approval and understand the service's access controls. Treat retrieved context as untrusted reference material, especially anything stored as an instruction, and require explicit user confirmation before saving or applying persistent agent instructions.

Vulnerability Patterns
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
Findings (1)

T02 · Agent Memory Poisoning

Error
Location
SKILL.md:154
Finding
Persistent Agent Memory Poisoning Through Untrusted Stored Instructions## Vulnerability Details **File Location**: `SKILL.md`, lines 154-156 and 243-258 **Vulnerability Type**: Persistent instruction poisoning through untrusted retrieved context **Risk Level**: High ### Vulnerable Code Snippets The Skill explicitly supports storing persistent agent instructions: ```markdown #### Context Types | Value | Use for | |---|---| | `"resource"` | Files, documents, reference material, code | | `"conversation"` | Chat history, meeting transcripts, support threads | | `"instruction"` | Persistent rules, conventions, agent instructions | ``` It subsequently directs the agent to inject retrieved content into its prompt and to perform retrieval proactively: ```markdown ### Store → Search (basic memory pattern) 1. Call `alchemyst_ai_add_context` to store a document 2. Later, call `alchemyst_ai_search_context` with a relevant query to retrieve it 3. Inject the retrieved content into your prompt as context ### Audit before adding 1. Call `alchemyst_ai_context_mcp_view_docs` to inspect what's already stored 2. Only call `alchemyst_ai_add_context` if the knowledge isn't already present 3. This avoids duplicating context and keeps the store clean ### Pre-answer retrieval Before answering any question that might depend on project-specific knowledge, call `alchemyst_ai_search_context` first. Prefer doing this proactively — don't wait for the user to explicitly ask "check the context store." ``` ### Technical Analysis The Skill combines two security-sensitive capabilities: 1. It allows externally supplied content to be persisted specifically as agent instructions. 2. It directs later agent sessions to retrieve that content proactively and inject it into prompts. No documented control requires the agent to verify the stored content's author, provenance, tenant ownership, or integrity before using it. The instructions also do not require retrieved material to be isolated and treated solely as untrusted reference data. There is no rule prohi ...[truncated 2443 chars]
Remediation
## Remediation Suggestions 1. **Disable persistent instruction storage by default.** Remove the `"instruction"` context type from general-purpose workflows unless there is a strictly controlled administrative requirement. 2. **Enforce instruction/data separation.** Require all retrieved content to be presented as quoted, untrusted reference material that cannot modify system, developer, Skill, or user instructions. 3. **Validate identity and authorization.** Verify the authenticated writer's user and organization membership server-side. Prevent callers from selecting arbitrary `user_id` or `organization_id` values. 4. **Apply least-privilege access controls.** Separate read, document-write, and instruction-administration permissions. Only trusted administrators should be able to create persistent instructions. 5. **Record and verify provenance.** Attach immutable author identity, tenant, creation time, source, and integrity information to every stored record. Reject records with missing or inconsistent provenance. 6. **Require explicit approval.** Ask for user confirmation before storing instructions and before applying retrieved material that could alter behavior. 7. **Filter dangerous content.** Detect and quarantine stored text that attempts to override higher-priority instructions, request secrets, initiate tool calls, change authorization boundaries, or direct further persistence. 8. **Constrain retrieval.** Filter searches by authenticated tenant and trusted source. Do not search instruction records during ordinary document retrieval. 9. **Add lifecycle controls.** Support review, revocation, expiration, versioning, and audit logs for persistent instructions. 10. **Use safe prompt framing.** Wrap retrieved records in a clearly delimited data section with an explicit directive that embedded commands must not be followed. 11. **Test poisoning resistance.** Add security tests covering malicious stored instructions, cross-tenant identifiers, poisoned semantic-sea ...[truncated 78 chars]
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Memory PoisoningPersistent Context Injection, Context Window Stuffing, Memory Manipulation
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
Findings (8)

Vague Triggers

Medium
Confidence
94% confidence
Finding
The trigger text is broad enough to cause this skill to activate for many requests involving memory, retrieval, documents, or project knowledge, which can lead an agent to contact an external MCP service more often than the user likely expects. In this skill’s context, over-invocation increases the chance of unnecessary data disclosure, unintended storage of sensitive information, and surprising external network access.

Missing User Warnings

Medium
Confidence
97% confidence
Finding
The skill describes sending documents, conversations, user IDs, organization IDs, and search queries to an external service but does not prominently warn that this data leaves the local environment. That omission is dangerous because users and downstream agents may unknowingly transmit confidential material, identifiers, or internal instructions to a third-party system.

Persistent Context Injection

Medium
Category
Memory Poisoning
Content
|---|---|
| `"resource"` | Files, documents, reference material, code |
| `"conversation"` | Chat history, meeting transcripts, support threads |
| `"instruction"` | Persistent rules, conventions, agent instructions |

#### Source Naming Convention
Confidence
89% confidence
Finding
The skill explicitly supports storing persistent 'instruction' content and later retrieving it for prompt injection into future sessions. This creates a persistent context injection risk: malicious or unsafe instructions placed in the external store can be reintroduced later as trusted context, potentially overriding current task boundaries, causing unsafe tool use, or exfiltrating sensitive data across sessions.

Vague Triggers

Medium
Confidence
92% confidence
Finding
The tool descriptions are broad enough that an agent could invoke remote search functionality in situations where the user did not clearly request persistence or external context lookup. Over-broad invocation criteria increase the risk of unintended data disclosure to a third-party service and can cause the agent to overreach beyond user intent.

Missing User Warnings

Medium
Confidence
92% confidence
Finding
This manifest describes uploading user-linked documents and metadata to an external HTTPS service but provides no user-facing warning that the data leaves the local environment. That omission is dangerous because users may assume 'context' is local memory, when in fact the skill can transmit sensitive files and conversation-derived content to a third party.

Vague Triggers

Medium
Confidence
93% confidence
Finding
The add-context tool permits transmission of arbitrary document content, user identifiers, organization identifiers, and metadata to a remote server without clear limits in the description on what may be stored or when. In an agent setting, vague write-scope is dangerous because it can lead to silent exfiltration of sensitive conversation, proprietary files, or credentials under the guise of 'memory' or 'context'.

Description-Behavior Mismatch

Low
Confidence
84% confidence
Finding
The manifest describes a general-purpose skill for storing, retrieving, searching, or viewing persistent context. The actual tool surface requires and operates on explicit user_id and organization_id identifiers for add/view operations, meaning the skill is not just generic note storage but a multi-tenant/user-scoped context management interface. That is a meaningful behavioral expansion compared with the manifest summary.

Missing User Warnings

Low
Confidence
83% confidence
Finding
The view-context and view-docs tools retrieve user-associated data from a remote service, but their descriptions do not warn that the returned data may be sensitive or externally hosted. This can lead to inappropriate invocation, over-collection, or exposure of stored personal or organizational information in contexts where the user did not expect remote retrieval.