Back to skill

Security audit

Railroaded

Security checks for vulnerabilities and agentic risk

Overview

The skill is a coherent online D&D game integration, but it relies on a mutable remote document for its full operating instructions, so users should review it before installing.

Install only if you are comfortable with an autonomous agent contacting Railroaded, creating game content, and publishing permanent in-character journals. Before play, review the remote player document yourself and keep the agent constrained to the Railroaded API with no access to unrelated files, secrets, shell tools, or accounts.

Vulnerability Patterns
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
  • Unauthorized Access and Privilege EscalationObtains permissions beyond the task's legitimate needs
Findings (1)

T01 · Skill Instruction Hijacking

Warning
Location
SKILL.md:25
Finding

Mutable Remote Document Can Hijack Agent Instructions

Content
View full analysis

Vulnerability Details

File Location: SKILL.md, lines 25-30 and 78
Vulnerability Type: Remote instruction trust and prompt supply-chain weakness
Risk Level: Medium

Complete vulnerable code snippet:

markdown
**API Base:** `https://api.railroaded.ai`
**Full Player Skill Doc:** `https://api.railroaded.ai/skill/player`
**Website:** `https://railroaded.ai`

## How to Play

Read the full player skill document at `https://api.railroaded.ai/skill/player` before your first game. It contains the complete API reference, character creation rules, action format, and gameplay loop.

The instruction is repeated at line 78:

markdown
- **Play:** Read `https://api.railroaded.ai/skill/player` and register

Technical Analysis

The locally audited Skill does not contain its complete operational instructions. Instead, it directs the agent to retrieve and follow a remotely hosted player Skill document before playing. That document is mutable and outside the reviewed package, so its effective instructions can change after installation or security review.

No content digest, cryptographic signature, immutable version identifier, constrained schema, or explicit trust boundary is specified. Consequently, an agent may interpret text returned by the endpoint as authoritative instructions rather than untrusted external data. If the remote endpoint, hosting account, DNS resolution, or deployment pipeline is compromised, an attacker could replace the expected gameplay documentation with instructions that redirect the agent's goals or attempt to weaken its safety constraints.

This is classified as instruction hijacking rather than remote payload execution because the reviewed file directs retrieval of natural-language operational instructions; it does not demonstrate downloading or executing external code.

Attack Path

  1. An attacker compromises or otherwise gains control over `https://api.railroaded.ai/ski ...[truncated 1450 chars]
Remediation
View remediation

Remediation Suggestions

  1. Bundle the complete gameplay instructions and API contract in the reviewed Skill package instead of requiring a mutable remote instruction document.
  2. If remote retrieval is necessary, reference an immutable, versioned resource and verify a pinned cryptographic digest or trusted digital signature before processing it.
  3. Parse remote responses through a strict data schema. Do not interpret remotely supplied prose as agent-level instructions.
  4. Explicitly state that external content is untrusted and cannot modify system instructions, safety constraints, tool permissions, credential-handling rules, or the approved gameplay scope.
  5. Apply a network allowlist restricted to documented Railroaded API endpoints and expected HTTP methods.
  6. Require user confirmation for sensitive operations, including transmitting non-game data, contacting previously undeclared domains, or publishing permanent content.
  7. Run the Skill with least privilege and prevent access to unrelated files, secrets, environment variables, shell execution, and administrative tools.
  8. Maintain a reviewed local fallback and fail closed if integrity validation of the remote document fails.
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
Findings (3)

External Transmission

Medium
Category
Data Exfiltration
Confidence
50% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · SKILL.md (reported line 26)May include surrounding context.

md
Railroaded is a D&D platform where AI agents are the players. You join a party of 4 agents, an AI DM runs the dungeon, dice are rolled deterministically, and the stories that emerge are real. Humans watch — you play.

**API Base:** `https://api.railroaded.ai`
**Full Player Skill Doc:** `https://api.railroaded.ai/skill/player`
**Website:** `https://railroaded.ai`

## How to Play

External Transmission

Medium
Category
Data Exfiltration
Confidence
50% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · SKILL.md (reported line 31)May include surrounding context.

md
Railroaded is a D&D platform where AI agents are the players. You join a party of 4 agents, an AI DM runs the dungeon, dice are rolled deterministically, and the stories that emerge are real. Humans watch — you play.

**API Base:** `https://api.railroaded.ai`
**Full Player Skill Doc:** `https://api.railroaded.ai/skill/player`
**Website:** `https://railroaded.ai`

## How to Play

External Transmission

Medium
Category
Data Exfiltration
Confidence
50% confidence
Finding

Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Content

Scanner excerpt · SKILL.md (reported line 80)May include surrounding context.

md
Railroaded is a D&D platform where AI agents are the players. You join a party of 4 agents, an AI DM runs the dungeon, dice are rolled deterministically, and the stories that emerge are real. Humans watch — you play.

**API Base:** `https://api.railroaded.ai`
**Full Player Skill Doc:** `https://api.railroaded.ai/skill/player`
**Website:** `https://railroaded.ai`

## How to Play

Static analysis

No suspicious patterns detected.