Numerai Tournament

• Outcome: Review
• Latest audit: Sep 11, 2026 · 1:07 AM UTC
• Version: 1.0.0

Overview

This skill is not overtly malicious, but it needs Review because it can use stored Numerai API keys to submit predictions or models in a cryptocurrency tournament where money can be lost.

Install only if you are comfortable granting an agent access to Numerai credentials and authenticated submission capability. Prefer environment variables or a secret manager over a long-lived credentials file, use a model with zero stake until you intentionally enable staking, review each upload before it happens, pin dependencies, and only unpickle model files you created in a trusted workspace.

A.I.G

Vulnerability Patterns

• Insecure Dependencies: Introduces malicious components through unsafe dependency sources
• Insecure Skill Coding Practices: Finds exploitable flaws such as hardcoded secrets or command injection
• Skill Instruction Hijacking: Alters the agent's session goals or safety constraints when the skill loads
• Agent Memory Poisoning: Writes attacker-controlled rules into memory that affect later sessions
• Remote Payload Retrieval and Execution: Fetches external code whose behavior can change after review

Findings (2)

T08 · Insecure Dependencies

Warning

• Location: SKILL.md:61

Finding

Unpinned Third-Party Dependencies Create a Supply-Chain Risk

Content

View full analysis

Vulnerability Details

File Location: SKILL.md, line 61
Vulnerability Type: Unpinned executable dependencies
Risk Level: Medium

Vulnerable Code

bash

pip install numerapi lightgbm pandas numpy cloudpickle scikit-learn

Technical Analysis

The installation command retrieves six packages without specifying reviewed versions, cryptographic hashes, or a lockfile. Consequently, the exact code installed can change between executions without any modification to the audited Skill.

Python source distributions can execute package-controlled build logic during installation. Malicious code in a wheel or source distribution can also execute when the installed package is subsequently imported by the documented workflow. A compromised upstream release, maliciously replaced distribution, or other package-index supply-chain incident could therefore introduce code not present during this audit.

The packages are installed inside a virtual environment, but a Python virtual environment is not a security sandbox. Package-controlled code still runs with the operating-system permissions of the invoking user.

Attack Path

1. An attacker compromises an upstream package release or its package-publishing account.
2. The attacker publishes a malicious version under one of the dependency names used by the Skill.
3. A user follows the documented unpinned pip install command.
4. pip resolves the compromised version because no known-safe version or hash is required.
5. Malicious code executes during package building, installation, or a later package import.
6. The code accesses files and environment variables available to the user, potentially including Numerai credentials.

Impact Assessment

Successful exploitation provides arbitrary code execution with the privileges of the user running the installation or subsequent Python workflow. The affected scope can include:

• NUMERAI_PUBLIC_ID and `NUMERAI_SEC ...[truncated 487 chars]

Remediation

View remediation

Remediation Suggestions

1. Replace the inline installation command with a reviewed requirements or lock file containing exact versions.

2. Record cryptographic hashes for every accepted distribution, including transitive dependencies.

3. Install with hash enforcement, for example:

   bash

   python3 -m pip install --require-hashes -r requirements.txt
   

4. Generate the lock file from a trusted environment and review dependency changes before updating it.

5. Prefer binary wheels from the official package index and reject unexpected source builds where practical.

6. Run installation and model processing in an isolated, non-privileged environment without unnecessary credentials.

7. Keep Numerai credentials out of the environment during dependency installation.

T09 · Insecure Skill Coding Practices

Warning

• Location: SKILL.md:237

Finding

Unsafe Deserialization of a Pickle Model File

Content

View full analysis

Vulnerability Details

File Location: SKILL.md, lines 237-239
Vulnerability Type: Unsafe Python pickle deserialization
Risk Level: Medium

Vulnerable Code

python

with open("models/ensemble_models.pkl", "rb") as f:
    sklearn_models = pickle.load(f)

Technical Analysis

Python pickle is an executable serialization format. A crafted pickle can invoke attacker-selected callables during pickle.load(); it does not need to wait for any method on the resulting object to be called.

The documented workflow normally creates models/ensemble_models.pkl locally. However, the loading operation does not verify the file's ownership, provenance, permissions, or cryptographic integrity. If another process, user, extracted archive, compromised dependency, or repository artifact can replace the file before this step, loading it can execute arbitrary Python code.

Restricting the expected object to scikit-learn models does not mitigate the issue because pickle instructions execute before the application can validate the deserialized object's type.

Attack Path

1. An attacker gains the ability to replace or supply models/ensemble_models.pkl. This could occur through a malicious shared project artifact, writable workspace, compromised model archive, or previously compromised dependency.
2. The attacker constructs a pickle whose reduction instructions invoke an operating-system command or malicious Python callable.
3. The user runs the documented model-upload build procedure.
4. pickle.load(f) interprets the malicious instructions immediately.
5. The payload executes with the same privileges and environment as the user running Python.
6. The payload can read credentials, alter generated models or predictions, perform authenticated API actions, or access other user-readable resources.

Impact Assessment

Successful exploitation results in arbitrary local code execution under the invoking user's ...[truncated 625 chars]

Remediation

View remediation

Remediation Suggestions

1. Avoid pickle for persisted model inputs. Store each LightGBM model using its native non-pickle representation, such as Booster.save_model(), and reconstruct the ensemble from explicitly parsed metadata.
2. Keep feature names and target mappings in a data-only format such as JSON, validating their schema before use.
3. If pickle cannot be eliminated, accept only files created by the same trusted workflow and never load downloaded or user-supplied pickle files.
4. Record and verify a trusted cryptographic digest before deserialization.
5. Ensure the model directory is owned by the invoking user and is not writable by untrusted users or processes.
6. Run unavoidable deserialization in a disposable, network-restricted sandbox without Numerai credentials or access to sensitive host paths.
7. Do not treat post-load type checks or restricted naming conventions as sufficient protection; they cannot prevent execution during unpickling.

SkillSpector

Vulnerability Patterns

• Privilege Escalation: Excessive Permissions, Sudo/Root Execution, Credential Access
• Rogue Agent: Self-Modification, Session Persistence
• Trigger Abuse: Overly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
• Prompt Injection: Instruction Override, Hidden Instructions, Exfiltration Commands
• Data Exfiltration: External Transmission, Env Variable Harvesting, File System Enumeration

Findings (8)

Credential Access

High

• Category: Privilege Escalation
• Confidence: 86% confidence

Finding

The skill instructs users to place long-lived API credentials in a predictable local file under the home directory. While common, this increases exposure to accidental disclosure, inclusion in backups, or unintended access by other local processes or agent tooling, especially in autonomous environments.

Content

Scanner excerpt · SKILL.md (reported line 44)May include surrounding context.

md

# Then create API keys at https://numer.ai/account
# Store credentials:
mkdir -p ~/.numerai
cat > ~/.numerai/credentials.json << 'CREDS'
{
  "public_id": "YOUR_PUBLIC_ID",
  "secret_key": "YOUR_SECRET_KEY"

Credential Access

High

• Category: Privilege Escalation
• Confidence: 70% confidence

Finding

Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Content

Scanner excerpt · SKILL.md (reported line 50)May include surrounding context.

"secret_key": "YOUR_SECRET_KEY" } CREDS chmod 600 ~/.numerai/credentials.json

text

Alternatively, set environment variables:

Credential Access

High

• Category: Privilege Escalation
• Confidence: 70% confidence

Finding

Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Content

Scanner excerpt · SKILL.md (reported line 315)May include surrounding context.

"secret_key": "YOUR_SECRET_KEY" } CREDS chmod 600 ~/.numerai/credentials.json

text

Alternatively, set environment variables:

Credential Access

High

• Category: Privilege Escalation
• Confidence: 94% confidence

Finding

The code reads API secrets from a local plaintext credentials file and uses them to authenticate outbound requests. In addition, the example uses with open('/.numerai/credentials.json'), which does not expand '' in Python and may cause developers to work around it unsafely; more importantly, direct file-based secret loading encourages local secret exposure in an automated skill context.

Content

Scanner excerpt · SKILL.md (reported line 195)May include surrounding context.

md

import json
from numerapi import NumerAPI

with open("~/.numerai/credentials.json") as f:
    creds = json.load(f)

napi = NumerAPI(creds["public_id"], creds["secret_key"])

Vague Triggers

Medium

• Category: Not specified by scanner
• Confidence: 87% confidence

Finding

The skill is framed as 'autonomous' tournament participation and earnings generation without clear activation boundaries, approval steps, or restrictions on when submissions should occur. In an agent setting, that broad wording can enable unintended execution of networked actions, model uploads, or financial participation with real credentials and possible NMR loss.

Content

No source excerpt is available for this finding.

Session Persistence

Medium

• Category: Rogue Agent
• Confidence: 83% confidence

Finding

The setup directs users to create and persist API credentials under ~/.numerai, establishing durable authentication material on disk for future automated use. In an autonomous skill that performs remote submissions and potentially financial actions, persistent session material increases the risk of unauthorized reuse or unintended repeated actions if the agent or environment is compromised.

Content

Scanner excerpt · SKILL.md (reported line 37)May include surrounding context.

Setup

1. Create a Numerai Account

bash

# Visit https://numer.ai to sign up

Sudo/Root Execution

Medium

• Category: Privilege Escalation
• Confidence: 80% confidence

Finding

Commands invoke sudo or root privileges. Verify this elevated access is necessary and justified.

Content

Scanner excerpt · SKILL.md (reported line 50)May include surrounding context.

"secret_key": "YOUR_SECRET_KEY" } CREDS chmod 600 ~/.numerai/credentials.json

text

Alternatively, set environment variables:

Sudo/Root Execution

Medium

• Category: Privilege Escalation
• Confidence: 80% confidence

Finding

Commands invoke sudo or root privileges. Verify this elevated access is necessary and justified.

Content

Scanner excerpt · SKILL.md (reported line 315)May include surrounding context.

md

- Your `NUMERAI_PUBLIC_ID` and `NUMERAI_SECRET_KEY` are sent to `api.numer.ai` for authentication
- Predictions (stock return rankings) are uploaded to Numerai's servers
- No other data leaves your machine
- Store credentials in `~/.numerai/credentials.json` with `chmod 600` permissions

Static analysis

No suspicious patterns detected.