Numerai Tournament

• Outcome: Review
• Latest audit: Sep 11, 2026 · 1:07 AM UTC
• Version: 1.0.0

Overview

This skill is not overtly malicious, but it needs Review because it can use stored Numerai API keys to submit predictions or models in a cryptocurrency tournament where money can be lost.

Install only if you are comfortable granting an agent access to Numerai credentials and authenticated submission capability. Prefer environment variables or a secret manager over a long-lived credentials file, use a model with zero stake until you intentionally enable staking, review each upload before it happens, pin dependencies, and only unpickle model files you created in a trusted workspace.

A.I.G

Vulnerability Patterns

• Insecure Dependencies: Introduces malicious components through unsafe dependency sources
• Insecure Skill Coding Practices: Finds exploitable flaws such as hardcoded secrets or command injection
• Skill Instruction Hijacking: Alters the agent's session goals or safety constraints when the skill loads
• Agent Memory Poisoning: Writes attacker-controlled rules into memory that affect later sessions
• Remote Payload Retrieval and Execution: Fetches external code whose behavior can change after review

Findings (2)

T08 · Insecure Dependencies

Warning

• Location: SKILL.md:61
• Finding: Unpinned Third-Party Dependencies Create a Supply-Chain Risk## Vulnerability Details **File Location**: `SKILL.md`, line 61 **Vulnerability Type**: Unpinned executable dependencies **Risk Level**: Medium ### Vulnerable Code ```bash pip install numerapi lightgbm pandas numpy cloudpickle scikit-learn ``` ### Technical Analysis The installation command retrieves six packages without specifying reviewed versions, cryptographic hashes, or a lockfile. Consequently, the exact code installed can change between executions without any modification to the audited Skill. Python source distributions can execute package-controlled build logic during installation. Malicious code in a wheel or source distribution can also execute when the installed package is subsequently imported by the documented workflow. A compromised upstream release, maliciously replaced distribution, or other package-index supply-chain incident could therefore introduce code not present during this audit. The packages are installed inside a virtual environment, but a Python virtual environment is not a security sandbox. Package-controlled code still runs with the operating-system permissions of the invoking user. ### Attack Path 1. An attacker compromises an upstream package release or its package-publishing account. 2. The attacker publishes a malicious version under one of the dependency names used by the Skill. 3. A user follows the documented unpinned `pip install` command. 4. `pip` resolves the compromised version because no known-safe version or hash is required. 5. Malicious code executes during package building, installation, or a later package import. 6. The code accesses files and environment variables available to the user, potentially including Numerai credentials. ### Impact Assessment Successful exploitation provides arbitrary code execution with the privileges of the user running the installation or subsequent Python workflow. The affected scope can include: - `NUMERAI_PUBLIC_ID` and `NUMERAI_SEC ...[truncated 487 chars]
• Remediation: ## Remediation Suggestions 1. Replace the inline installation command with a reviewed requirements or lock file containing exact versions. 2. Record cryptographic hashes for every accepted distribution, including transitive dependencies. 3. Install with hash enforcement, for example: ```bash python3 -m pip install --require-hashes -r requirements.txt ``` 4. Generate the lock file from a trusted environment and review dependency changes before updating it. 5. Prefer binary wheels from the official package index and reject unexpected source builds where practical. 6. Run installation and model processing in an isolated, non-privileged environment without unnecessary credentials. 7. Keep Numerai credentials out of the environment during dependency installation.

T09 · Insecure Skill Coding Practices

Warning

• Location: SKILL.md:237
• Finding: Unsafe Deserialization of a Pickle Model File## Vulnerability Details **File Location**: `SKILL.md`, lines 237-239 **Vulnerability Type**: Unsafe Python pickle deserialization **Risk Level**: Medium ### Vulnerable Code ```python with open("models/ensemble_models.pkl", "rb") as f: sklearn_models = pickle.load(f) ``` ### Technical Analysis Python pickle is an executable serialization format. A crafted pickle can invoke attacker-selected callables during `pickle.load()`; it does not need to wait for any method on the resulting object to be called. The documented workflow normally creates `models/ensemble_models.pkl` locally. However, the loading operation does not verify the file's ownership, provenance, permissions, or cryptographic integrity. If another process, user, extracted archive, compromised dependency, or repository artifact can replace the file before this step, loading it can execute arbitrary Python code. Restricting the expected object to scikit-learn models does not mitigate the issue because pickle instructions execute before the application can validate the deserialized object's type. ### Attack Path 1. An attacker gains the ability to replace or supply `models/ensemble_models.pkl`. This could occur through a malicious shared project artifact, writable workspace, compromised model archive, or previously compromised dependency. 2. The attacker constructs a pickle whose reduction instructions invoke an operating-system command or malicious Python callable. 3. The user runs the documented model-upload build procedure. 4. `pickle.load(f)` interprets the malicious instructions immediately. 5. The payload executes with the same privileges and environment as the user running Python. 6. The payload can read credentials, alter generated models or predictions, perform authenticated API actions, or access other user-readable resources. ### Impact Assessment Successful exploitation results in arbitrary local code execution under the invoking user's ...[truncated 625 chars]
• Remediation: ## Remediation Suggestions 1. Avoid pickle for persisted model inputs. Store each LightGBM model using its native non-pickle representation, such as `Booster.save_model()`, and reconstruct the ensemble from explicitly parsed metadata. 2. Keep feature names and target mappings in a data-only format such as JSON, validating their schema before use. 3. If pickle cannot be eliminated, accept only files created by the same trusted workflow and never load downloaded or user-supplied pickle files. 4. Record and verify a trusted cryptographic digest before deserialization. 5. Ensure the model directory is owned by the invoking user and is not writable by untrusted users or processes. 6. Run unavoidable deserialization in a disposable, network-restricted sandbox without Numerai credentials or access to sensitive host paths. 7. Do not treat post-load type checks or restricted naming conventions as sufficient protection; they cannot prevent execution during unpickling.

SkillSpector

Vulnerability Patterns

• Privilege Escalation: Excessive Permissions, Sudo/Root Execution, Credential Access
• Rogue Agent: Self-Modification, Session Persistence
• Trigger Abuse: Overly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
• Prompt Injection: Instruction Override, Hidden Instructions, Exfiltration Commands
• Data Exfiltration: External Transmission, Env Variable Harvesting, File System Enumeration

Findings (8)

Credential Access

High

• Category: Privilege Escalation

Content

# Then create API keys at https://numer.ai/account
# Store credentials:
mkdir -p ~/.numerai
cat > ~/.numerai/credentials.json << 'CREDS'
{
  "public_id": "YOUR_PUBLIC_ID",
  "secret_key": "YOUR_SECRET_KEY"

• Confidence: 86% confidence
• Finding: The skill instructs users to place long-lived API credentials in a predictable local file under the home directory. While common, this increases exposure to accidental disclosure, inclusion in backups, or unintended access by other local processes or agent tooling, especially in autonomous environments.

Credential Access

High

• Category: Privilege Escalation

Content

"secret_key": "YOUR_SECRET_KEY"
}
CREDS
chmod 600 ~/.numerai/credentials.json
```

Alternatively, set environment variables:

• Confidence: 70% confidence
• Finding: Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Credential Access

High

• Category: Privilege Escalation

Content

"secret_key": "YOUR_SECRET_KEY"
}
CREDS
chmod 600 ~/.numerai/credentials.json
```

Alternatively, set environment variables:

• Confidence: 70% confidence
• Finding: Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Credential Access

High

• Category: Privilege Escalation

Content

import json
from numerapi import NumerAPI

with open("~/.numerai/credentials.json") as f:
    creds = json.load(f)

napi = NumerAPI(creds["public_id"], creds["secret_key"])

• Confidence: 94% confidence
• Finding: The code reads API secrets from a local plaintext credentials file and uses them to authenticate outbound requests. In addition, the example uses with open('~/.numerai/credentials.json'), which does not expand '~' in Python and may cause developers to work around it unsafely; more importantly, direct file-based secret loading encourages local secret exposure in an automated skill context.

Vague Triggers

Medium

• Confidence: 87% confidence
• Finding: The skill is framed as 'autonomous' tournament participation and earnings generation without clear activation boundaries, approval steps, or restrictions on when submissions should occur. In an agent setting, that broad wording can enable unintended execution of networked actions, model uploads, or financial participation with real credentials and possible NMR loss.

Session Persistence

Medium

• Category: Rogue Agent

Content

## Setup

### 1. Create a Numerai Account

```bash
# Visit https://numer.ai to sign up

• Confidence: 83% confidence
• Finding: The setup directs users to create and persist API credentials under ~/.numerai, establishing durable authentication material on disk for future automated use. In an autonomous skill that performs remote submissions and potentially financial actions, persistent session material increases the risk of unauthorized reuse or unintended repeated actions if the agent or environment is compromised.

Sudo/Root Execution

Medium

• Category: Privilege Escalation

Content

"secret_key": "YOUR_SECRET_KEY"
}
CREDS
chmod 600 ~/.numerai/credentials.json
```

Alternatively, set environment variables:

• Confidence: 80% confidence
• Finding: Commands invoke sudo or root privileges. Verify this elevated access is necessary and justified.

Sudo/Root Execution

Medium

• Category: Privilege Escalation

Content

- Your `NUMERAI_PUBLIC_ID` and `NUMERAI_SECRET_KEY` are sent to `api.numer.ai` for authentication
- Predictions (stock return rankings) are uploaded to Numerai's servers
- No other data leaves your machine
- Store credentials in `~/.numerai/credentials.json` with `chmod 600` permissions

• Confidence: 80% confidence
• Finding: Commands invoke sudo or root privileges. Verify this elevated access is necessary and justified.

Static analysis

No suspicious patterns detected.