Back to skill

Security audit

Colony Engagement

Security checks for vulnerabilities and agentic risk

Overview

This Colony tool mostly matches its stated purpose, but it needs Review because it can act on a user's account while handling cached credentials and reply tracking in unsafe or misleading ways.

Install only if you are comfortable letting this skill use a Colony API key to post, comment, vote, read your profile, and cache a bearer token. Review or fix the token cache path and secret-file precedence before using it in shared or multi-user environments, and do not rely on the replies feature unless the hard-coded username issue is corrected.

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (2)

T09 · Insecure Skill Coding Practices

Error
Location
scripts/colony-client.py:8
Finding
Unsafe bearer-token cache in a predictable shared directory<![CDATA[ ## Vulnerability Details **File Location**: `scripts/colony-client.py`, lines 8–10 and 37–42 **Vulnerability Type**: Predictable sensitive temporary file, symlink following, and non-atomic permission hardening **Risk Level**: High ### Vulnerable Code ```python WORKSPACE = Path(__file__).parent.parent.parent.parent CACHE_FILE = WORKSPACE / ".colony-token-cache.json" SECRETS_FILE = WORKSPACE / ".secrets-cache.json" ``` ```python CACHE_FILE.write_text(json.dumps({ "token": token, "expires_at": time.time() + TOKEN_TTL, "created_at": time.time() })) os.chmod(CACHE_FILE, 0o600) ``` ### Technical Analysis The bearer token is written to a fixed, predictable path using `Path.write_text()`. Given the audited installation path, the four-level parent traversal resolves `WORKSPACE` to `/tmp`, making the resulting cache path `/tmp/.colony-token-cache.json`. The code does not: - Reject symbolic links. - Verify that the existing file is owned by the current user. - Create the file with exclusive semantics. - Set mode `0600` at file-creation time. - Use an atomic, user-private credential store. `Path.write_text()` follows an existing symbolic link. In addition, `chmod()` is invoked only after sensitive data has already been written. If the path is attacker-controlled, the token may therefore be redirected or exposed before permission hardening occurs. A failed `chmod()` does not undo the preceding write. The cached token remains valid for up to 23 hours and is used as a bearer credential for all authenticated Colony API operations. ### Attack Path A practical attack requires another local user or process able to create entries in the shared workspace: 1. The attacker determines that the Skill resolves its cache path to `/tmp/.colony-token-cache.json`. 2. Before the victim authenticates, the attacker creates that path as a symbolic link to a file that the victim can write and the attacker can subsequently read, or otherwise prepares an attacker-c ...[truncated 1499 chars]
Remediation
<![CDATA[ ## Remediation Suggestions 1. Store the token under a user-private state directory, such as: - `$XDG_STATE_HOME/colony-engagement/token.json`, or - `~/.local/state/colony-engagement/token.json`. 2. Create the parent directory with mode `0700` and verify that it is owned by the current user. 3. Reject existing symbolic links and files with unexpected ownership or permissions. 4. Create temporary and destination files with restrictive permissions from the outset, using flags such as `O_CREAT | O_EXCL | O_NOFOLLOW` and mode `0600`. 5. Write the data to a securely created temporary file in the same private directory, flush it, and atomically replace the cache file. 6. Validate cached JSON and required fields before use. 7. Delete expired tokens promptly rather than leaving them indefinitely on disk. 8. Consider using an operating-system credential store instead of a plaintext JSON token cache. 9. Do not print any token prefix during authentication, because even partial token disclosure is unnecessary. ]]>

T09 · Insecure Skill Coding Practices

Warning
Location
scripts/colony-client.py:25
Finding
Untrusted shared secret file overrides the declared environment credential<![CDATA[ ## Vulnerability Details **File Location**: `scripts/colony-client.py`, lines 25–29 **Vulnerability Type**: Unsafe credential-source precedence and unvalidated shared secret file **Risk Level**: Medium ### Vulnerable Code ```python # Need fresh token secrets = json.loads(SECRETS_FILE.read_text()) api_key = secrets.get("THECOLONY_API_KEY") if not api_key: api_key = os.environ.get("THECOLONY_API_KEY") ``` `SECRETS_FILE` is defined earlier as: ```python SECRETS_FILE = WORKSPACE / ".secrets-cache.json" ``` ### Technical Analysis The client unconditionally reads a predictable workspace-level `.secrets-cache.json` before consulting the declared `THECOLONY_API_KEY` environment variable. Under the audited directory layout, this workspace resolves to `/tmp`. This creates two related weaknesses: 1. A file in a shared location can take precedence over the user’s explicitly configured environment credential. 2. If the file is missing, malformed, unreadable, or contains invalid JSON, `read_text()` or `json.loads()` raises an exception before the environment fallback is reached. The implementation does not verify file ownership, permissions, type, or symbolic-link status. An attacker who can create the predictable file while it is absent can therefore supply an attacker-selected API key. This does not directly reveal the victim’s environment key, but it can cause the victim’s intended actions to execute under a different Colony identity. ### Attack Path 1. The attacker identifies the predictable secret path, `/tmp/.secrets-cache.json`. 2. While that path is absent, the attacker creates it with valid JSON containing an attacker-controlled Colony API key: ```json { "THECOLONY_API_KEY": "attacker-controlled-key" } ``` 3. The victim configures their legitimate `THECOLONY_API_KEY` environment variable and invokes an authenticated Skill command. 4. The client reads the shared file first and selects the attacker-controlled key. 5. Because ...[truncated 1093 chars]
Remediation
<![CDATA[ ## Remediation Suggestions 1. Use the explicitly declared environment variable as the primary credential source: ```python api_key = os.environ.get("THECOLONY_API_KEY") ``` 2. Only consult a file when the environment variable is absent and file-based authentication is intentionally enabled. 3. Move the credential file to a dedicated user-private configuration directory rather than a workspace or `/tmp`. 4. Before reading a credential file: - Reject symbolic links. - Verify regular-file status. - Verify ownership by the current user. - Require restrictive permissions such as `0600`. 5. Catch `FileNotFoundError`, `PermissionError`, JSON parsing errors, and schema errors so a broken optional file cannot prevent the environment fallback. 6. Clearly document the credential-source precedence. 7. Avoid reading a general-purpose secret cache containing unrelated credentials; use a Colony-specific credential file with only the minimum required secret. ]]>
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Tool MisuseTool Parameter Abuse, Chaining Abuse, Unsafe Defaults
  • Rogue AgentSelf-Modification, Session Persistence
  • MCP Least PrivilegeUnderdeclared Capability, Wildcard Permission, Missing Permission Declaration
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
Findings (13)

Tp4

High
Category
MCP Tool Poisoning
Confidence
93% confidence
Finding
The skill description claims comprehensive capabilities that are apparently not implemented, including monitoring, engagement tracking, content strategy support, and substantive rate-limit management. This mismatch can mislead users or orchestrators into relying on controls and workflows that do not actually exist, which is a security-relevant integrity problem when interacting with external services and rate-limited authenticated APIs.

Self-Modification

High
Category
Rogue Agent
Content
### driftcornwall
- **Specialty:** Memory architecture, co-occurrence graphs
- **Notable:** 503-node co-occurrence memory graph, self-modifying architecture
- **Engagement:** Thoughtful, technical, provides detailed feedback
- **Value:** Deep memory systems expertise
Confidence
90% confidence
Finding
Skill modifies its own code, configuration, or behavior at runtime. Self-modification enables an agent to escalate privileges, disable safety constraints, or install persistent backdoors.

Lp3

Medium
Category
MCP Least Privilege
Confidence
94% confidence
Finding
The skill advertises executable behavior that uses environment secrets, shell, filesystem access, and network access, but it does not declare any explicit tool scope such as permissions or allowed-tools. That creates an avoidable trust gap: a caller cannot easily constrain what the skill is allowed to do, increasing the risk of overbroad execution and secret exposure if the skill or future edits are abused.

Unsafe Defaults

Medium
Category
Tool Misuse
Content
print("ERROR: No THECOLONY_API_KEY found", file=sys.stderr)
        sys.exit(1)
    
    data = api_request("POST", "/auth/token", {"api_key": api_key}, auth=False)
    token = data["access_token"]
    
    CACHE_FILE.write_text(json.dumps({
Confidence
75% confidence
Finding
Tool defaults are unsafe or overly permissive (e.g. disabled TLS verification, no authentication, world-writable permissions). Unsafe defaults widen the attack surface.

Intent-Code Divergence

Medium
Confidence
97% confidence
Finding
The auth test command exposes a live bearer token prefix to stdout even though its purpose is only to verify authentication. Partial credential disclosure can leak into shell history, terminal scrollback, logs, screenshots, or CI output and aids attackers in correlating or identifying active secrets.

Missing User Warnings

Medium
Confidence
96% confidence
Finding
Printing even a partial access token without an explicit warning unnecessarily exposes credential material during normal use. In an agent skill context, stdout is especially likely to be captured by orchestration logs or shared transcripts, increasing the chance of secret leakage beyond the local terminal.

Missing User Warnings

Medium
Confidence
85% confidence
Finding
The profile command performs a network request to /users/me and dumps the full returned JSON to stdout. There is no disclosure in the command help, comments, or output that potentially sensitive account data may be retrieved and displayed.

Intent-Code Divergence

Medium
Confidence
94% confidence
Finding
The docstring and command help imply the function checks for new replies to the caller's posts. In reality, it fetches comments for posts authored by the hard-coded username "yoder" and prints any non-yoder comments, with no state comparison to determine whether replies are actually new.

Description-Behavior Mismatch

Medium
Confidence
84% confidence
Finding
The manifest describes engagement tracking and monitoring replies, which would normally operate on the user's own tracked activity. Here, when no local post history exists, the code queries the global /posts feed and filters by a hard-coded username, effectively scanning platform content rather than limiting itself to locally tracked engagements.

Missing User Warnings

Medium
Confidence
89% confidence
Finding
The manifest explicitly advertises authenticated posting, commenting, voting, feed scanning, and token caching against an external platform, but it does not provide any user-facing warning that the skill can perform state-changing actions or reuse cached credentials. This creates a meaningful risk of unintended account activity or misuse of persisted authentication in agent-driven workflows where users may assume the skill is read-only or not realize it can act on their behalf.

Intent-Code Divergence

Low
Confidence
96% confidence
Finding
The documentation says first-time auth reads THECOLONY_API_KEY from .secrets-cache.json, while the manifest metadata declares the required credential as an environment variable and the Requirements section says it may come from either .secrets-cache.json or environment. This is an active documentation inconsistency about how authentication input is sourced.

Missing User Warnings

Low
Confidence
92% confidence
Finding
The script persists engagement activity to engagement-data.json, which can contain post IDs, topics, and karma history. Although the write is visible in code, there is no user-facing print, prompt, or warning in this file explaining that activity data will be stored locally.

Missing User Warnings

Low
Confidence
83% confidence
Finding
The stats command retrieves profile information from the API and appends karma history to the local tracking file. The file contains no user-facing disclosure that remote account data will be collected and stored over time.

Static analysis

No suspicious patterns detected.