Back to skill

Security audit

Orb

Security checks for vulnerabilities and agentic risk

Overview

This skill mostly matches its hosted-artifact purpose, but it asks agents to persist an Orb API key in local agent and shell configuration files without clear user-controlled consent.

Install only if you are comfortable sending artifact content to Orb and managing an Orb API key. Do not let the agent write the key into agent configs or shell startup files unless you deliberately want persistent plaintext storage; prefer setting ORB_API_KEY only for the current session or through a secure secret manager.

Vulnerability Patterns
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (1)

T09 · Insecure Skill Coding Practices

Warning
Location
SKILL.md:37
Finding
Insecure Plaintext Persistence of Orb API Credentials<![CDATA[ ## Vulnerability Details **File Location**: `SKILL.md`, lines 37–49 **Vulnerability Type**: Plaintext credential storage and unsafe configuration modification **Risk Level**: Medium ### Vulnerable Code ```markdown Before making any API call, check that `$ORB_API_KEY` is set. If it is not: 1. Ask the user for their Orb API key: > I need your Orb API key to create artifacts. > Get one at **https://byorb.app/dashboard/settings** → **Create API Key** 2. Once the user provides the key, **persist it** so it survives across sessions. Use your native configuration mechanism: - **Claude Code**: Add to `~/.claude/settings.json` under `"env": { "ORB_API_KEY": "..." }` - **Gemini CLI**: Write to `~/.gemini/.env` as `ORB_API_KEY=...` - **Codex**: Add to `~/.codex/config.toml` under `[shell_environment_policy]` with `set = { ORB_API_KEY = "..." }` - **Fallback**: Append `export ORB_API_KEY=...` to the user's shell profile (`~/.zshrc` or `~/.bashrc`) 3. Also `export ORB_API_KEY=...` in the current session so it's available immediately. ``` ### Technical Analysis The Skill explicitly requires an Agent to persist `ORB_API_KEY` in plaintext configuration files, dotenv files, or shell startup files. Persistent storage is not required for the declared operation of submitting artifacts to the Orb API; a process-scoped or session-scoped environment variable is sufficient. The instructions do not require: - Explicit user consent before persistent storage. - Use of an operating-system credential manager or secret vault. - Restrictive owner-only permissions on credential files. - Validation or safe encoding of the supplied credential. - Protection against symlink attacks when writing configuration files. - Secure removal or API-key revocation procedures. The shell-profile fallback is particularly risky. Files such as `~/.zshrc` and `~/.bashrc` are interpreted whenever a new shell starts. If an Agent directly interpolates an untrusted key into a ...[truncated 2612 chars]
Remediation
<![CDATA[ ## Remediation Suggestions 1. **Use session-scoped credentials by default** - Accept `ORB_API_KEY` only from the existing process environment. - If absent, instruct the user to configure it outside the Skill or use it only for the current process. - Do not automatically modify persistent files. 2. **Require explicit consent for persistence** - Clearly identify the destination file, persistence duration, and disclosure risks. - Obtain confirmation before writing any credential to disk. 3. **Use a secure credential store** - Prefer an operating-system keychain, platform secret manager, or Agent-native encrypted credential facility. - Store only a secret reference in ordinary configuration files. 4. **Protect any unavoidable credential file** - Create it with owner-only permissions, such as mode `0600`. - Verify that the destination and its parent directories are owned by the expected user. - Reject symbolic links and use atomic file replacement where appropriate. - Avoid placing secrets in shell startup files. 5. **Prevent injection and configuration corruption** - Do not generate shell commands by concatenating the supplied key. - Use structured JSON or TOML serializers for configuration files. - If shell representation is unavoidable, validate the expected token format and apply robust shell escaping through a trusted mechanism. 6. **Support credential lifecycle management** - Document where the key is stored. - Provide removal and rotation instructions. - Recommend immediate revocation if the key may have been disclosed. ]]>
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
  • Rogue AgentSelf-Modification, Session Persistence
  • Trigger AbuseOverly Broad Trigger, Shadow Command Trigger, Keyword Baiting Trigger
  • MCP Tool PoisoningHidden Instructions, Unicode Deception, Parameter Description Injection
Findings (18)

Vague Triggers

High
Confidence
98% confidence
Finding
The top-level trigger phrases are broad enough to match many ordinary requests such as making something shareable, interactive, or creating a document/chart. That can cause unexpected activation of the skill and silent transmission of user content to an external service when the user may not have intended to invoke Orb.

Context-Inappropriate Capability

High
Confidence
99% confidence
Finding
The skill explicitly instructs the agent to persist a user-provided API key across sessions by modifying local config files and shell profiles. That exceeds the stated artifact-creation purpose, creates unnecessary long-term secret storage, and increases the risk of credential exposure, reuse by unrelated workflows, or compromise through other local tools reading those files.

Missing User Warnings

High
Confidence
98% confidence
Finding
The skill directs the agent to store a user API key persistently in configuration files and shell profiles without a clear warning that this changes the local environment and creates durable credential exposure. Users may unknowingly grant long-lived access to the Orb account beyond the immediate task.

Ssd 3

High
Confidence
99% confidence
Finding
Persisting a user-provided secret into dotfiles and agent configuration directories is a sensitive data handling violation. It expands the attack surface from one transient API call to long-term credential residency on disk, where other processes, backups, logs, or future sessions may access the secret.

Agent Config Directory Access

High
Category
Agent Snooping
Content
2. Once the user provides the key, **persist it** so it survives across sessions.
   Use your native configuration mechanism:
   - **Claude Code**: Add to `~/.claude/settings.json` under `"env": { "ORB_API_KEY": "..." }`
   - **Gemini CLI**: Write to `~/.gemini/.env` as `ORB_API_KEY=...`
   - **Codex**: Add to `~/.codex/config.toml` under `[shell_environment_policy]` with `set = { ORB_API_KEY = "..." }`
   - **Fallback**: Append `export ORB_API_KEY=...` to the user's shell profile (`~/.zshrc` or `~/.bashrc`)
Confidence
98% confidence
Finding
The skill specifically targets an agent configuration directory for modification, instructing the agent to write secrets into ~/.claude/settings.json. Accessing and altering agent config files is dangerous because it changes future runtime behavior and stores credentials in a location likely trusted by the agent environment.

Agent Config Directory Access

High
Category
Agent Snooping
Content
Use your native configuration mechanism:
   - **Claude Code**: Add to `~/.claude/settings.json` under `"env": { "ORB_API_KEY": "..." }`
   - **Gemini CLI**: Write to `~/.gemini/.env` as `ORB_API_KEY=...`
   - **Codex**: Add to `~/.codex/config.toml` under `[shell_environment_policy]` with `set = { ORB_API_KEY = "..." }`
   - **Fallback**: Append `export ORB_API_KEY=...` to the user's shell profile (`~/.zshrc` or `~/.bashrc`)

3. Also `export ORB_API_KEY=...` in the current session so it's available immediately.
Confidence
98% confidence
Finding
The skill instructs writing the API key into ~/.codex/config.toml, which is direct modification of the agent's configuration state. This creates persistence and broadens the blast radius if the config file is read by other tools, synced, backed up, or exposed through local compromise.

Vague Triggers

High
Confidence
99% confidence
Finding
The update triggers include extremely generic verbs like 'change', 'update', 'make it', 'fix', and 'add'. In conversation, these are common edit requests and could accidentally cause updates to a remotely hosted artifact instead of a local or inline response, leading to unintended external data transmission and modification.

Session Persistence

Medium
Category
Rogue Agent
Content
---
name: orb
description: >
  Create and share rich interactive artifacts (webpages, markdown, flowcharts)
  as hosted links from any AI conversation. Use when the user asks to create
  a dashboard, report, chart, visualization, interactive page, or any shareable
  content. Triggers on: "create an artifact", "make this shareable",
Confidence
60% confidence
Finding
Skill establishes unauthorized persistence across sessions via cron jobs, startup scripts, or state files. Session persistence allows an attacker to maintain access beyond the current interaction.

External Transmission

Medium
Category
Data Exfiltration
Content
# Orb — Artifacts for your AI Agent

Orb is a hosted artifact service. Use the API at https://api.byorb.app/v1.
Artifacts are rendered at https://art.byorb.app/v/<id>.

This skill uses the web API only. Do not save artifact content to disk.
Confidence
50% confidence
Finding
Data is being sent to an external URL. This could be legitimate telemetry or data exfiltration. Manual review is recommended.

Description-Behavior Mismatch

Medium
Confidence
92% confidence
Finding
The documentation claims the skill uses only the web API and does not save content to disk, but later instructs the agent to write credentials into local files. This inconsistency can mislead users and reviewers about the skill's actual behavior, reducing informed consent around secret handling and local system modification.

External Transmission

Medium
Category
Data Exfiltration
Content
visualization / webpage", "make this interactive", "make this shareable"

```bash
curl -sS -X POST https://api.byorb.app/v1/artifacts \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $ORB_API_KEY" \
  -d '{
Confidence
89% confidence
Finding
This finding duplicates the concrete POST-based data transmission behavior already identified. The security issue is not the URL string itself, but that user content is uploaded to a third-party hosted service under broad activation conditions.

External Transmission

Medium
Category
Data Exfiltration
Content
visualization / webpage", "make this interactive", "make this shareable"

```bash
curl -sS -X POST https://api.byorb.app/v1/artifacts \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $ORB_API_KEY" \
  -d '{
Confidence
89% confidence
Finding
This finding duplicates the concrete POST-based data transmission behavior already identified. The security issue is not the URL string itself, but that user content is uploaded to a third-party hosted service under broad activation conditions.

External Transmission

Medium
Category
Data Exfiltration
Content
Use the `id` from this conversation. The URL stays the same.

```bash
curl -sS -X PUT https://api.byorb.app/v1/artifacts/<id> \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $ORB_API_KEY" \
  -d '{"content": "<updated full content>"}'
Confidence
89% confidence
Finding
This duplicates the update-path transmission risk: full artifact contents are sent to Orb during PUT operations. In a hosting skill this is expected functionality, but accidental triggering can still leak sensitive information externally.

External Transmission

Medium
Category
Data Exfiltration
Content
Use the `id` from this conversation. The URL stays the same.

```bash
curl -sS -X PUT https://api.byorb.app/v1/artifacts/<id> \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $ORB_API_KEY" \
  -d '{"content": "<updated full content>"}'
Confidence
89% confidence
Finding
This duplicates the update-path transmission risk: full artifact contents are sent to Orb during PUT operations. In a hosting skill this is expected functionality, but accidental triggering can still leak sensitive information externally.

External Transmission

Medium
Category
Data Exfiltration
Content
Step 1 — Fetch pending feedback:

```bash
curl -sS "https://api.byorb.app/v1/artifacts/<id>/feedback?status=pending" \
  -H "Authorization: Bearer $ORB_API_KEY"
```
Confidence
80% confidence
Finding
Fetching artifact feedback from the external service transmits artifact identifiers and account-scoped metadata to a third party. This is part of the feature set, but it still expands data exposure and should only occur when the user clearly requests checking remote feedback.

External Transmission

Medium
Category
Data Exfiltration
Content
Step 3 — Mark processed:

```bash
curl -sS -X PATCH https://api.byorb.app/v1/artifacts/<id>/feedback/processed \
  -H "Authorization: Bearer $ORB_API_KEY"
```
Confidence
80% confidence
Finding
Marking feedback as processed performs a state-changing external request on the user's hosted artifact. While expected in workflow terms, it should not happen automatically without clear user intent because it mutates remote state and may affect collaboration or auditability.

External Transmission

Medium
Category
Data Exfiltration
Content
When a Webpage artifact needs an image that cannot be SVG or base64:

```bash
curl -sS -X POST https://api.byorb.app/v1/assets \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $ORB_API_KEY" \
  -d '{"image_url": "https://external.com/photo.png"}'
Confidence
90% confidence
Finding
The asset-upload flow instructs the service to retrieve an external image URL, which can disclose user-selected URLs and import third-party content into Orb-hosted artifacts. This is functionally relevant, but it introduces privacy, provenance, and content-safety considerations.

External Transmission

Medium
Category
Data Exfiltration
Content
Triggers: "show my artifacts", "list my artifacts", "what have I created"

```bash
curl -sS https://api.byorb.app/v1/artifacts \
  -H "Authorization: Bearer $ORB_API_KEY"
```
Confidence
83% confidence
Finding
Listing artifacts queries the external service for the user's hosted content metadata. This is expected capability, but it still involves external account data access and should only be performed when the user explicitly asks for their Orb artifacts.

Static analysis

No suspicious patterns detected.