ClawHub
Back to skill

Security audit

Anyway Traces

Security checks for vulnerabilities and agentic risk

Overview

The skill is a coherent observability integration, but it directs broad automatic LLM telemetry to an external collector without clear user consent, scoping, or redaction guidance.

Before installing, confirm what Anyway collects by default, whether prompts and responses are captured, who can access exported traces, and how retention/deletion works. Use it only in projects where external observability export is acceptable, and configure redaction or narrower instrumentation for sensitive workloads.

A.I.G

Vulnerability Patterns
  • Insecure DependenciesIntroduces malicious components through unsafe dependency sources
  • Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Findings (2)

T09 · Insecure Skill Coding Practices

Warning
Location
skills.md:53
Finding

Unrestricted Automatic LLM Telemetry May Expose Sensitive Information

Content
View full analysis

Vulnerability Details

File Location: skills.md, lines 53-57
Vulnerability Type: Automatic sensitive-data transmission to an external telemetry collector
Risk Level: Medium

Vulnerable Code

markdown
- Never hardcode API keys — use environment variables or a secrets manager
- Always call `Traceloop.init()` (Python) or `initialize()` (JS) before any LLM calls
- For ESM/Next.js projects, always pass provider modules via `instrumentModules`
- The collector endpoint is `https://collector.anyway.sh/`
- The SDK auto-instruments calls to OpenAI, Anthropic, Cohere, Bedrock, Vertex AI, and other providers — manual span creation is only needed for workflow/task structure

Technical Analysis

The Skill requires the Anyway SDK to be initialized before any LLM calls and states that the SDK automatically instruments multiple LLM providers. It also configures an external collector at https://collector.anyway.sh/.

Automatic instrumentation is consistent with the Skill's declared observability purpose. However, the instructions do not require the agent to:

  • Obtain approval before exporting telemetry.
  • Review which trace attributes are collected.
  • Redact prompts, responses, authorization data, personal information, or business data.
  • Limit instrumentation to specific providers, functions, or environments.
  • Configure retention or access controls.
  • Provide a local-only or no-export mode for sensitive workloads.

Consequently, the required global instrumentation may exceed the minimum privileges necessary for observability. The reviewed file does not establish that credentials are transmitted, but it creates a credible route through which prompts, responses, metadata, identifiers, or other sensitive application data could leave the local environment.

Attack Path

  1. A user asks an agent to instrument an LLM application using this Skill.
  2. Following the mandatory rule, the agent initializes the observability SDK before any LLM call.

...[truncated 1242 chars]

Remediation
View remediation

Remediation Suggestions

  1. Require explicit user authorization before enabling external telemetry export.
  2. Document the exact fields collected by default, including whether prompts and responses are captured.
  3. Default to excluding prompt and response content unless the user explicitly opts in.
  4. Add mandatory filtering for API keys, authorization headers, cookies, tokens, personal data, and other sensitive attributes.
  5. Scope instrumentation to the minimum providers, functions, and environments needed for the requested task.
  6. Provide a local-only or no-export mode for development and sensitive workloads.
  7. Require transport encryption, tenant isolation, access controls, retention limits, and deletion procedures for exported traces.
  8. Instruct the agent to present the proposed collector endpoint and telemetry configuration for review before modifying an application.
  9. Add tests that confirm prohibited fields cannot be exported.

T08 · Insecure Dependencies

Warning
Location
skills.md:2
Finding

Security-Critical Instructions Are Delegated to Mutable Remote Documentation

Content
View full analysis

Vulnerability Details

File Location: skills.md, lines 2-3 and 39-47
Vulnerability Type: Unpinned external documentation and supply-chain instructions
Risk Level: Medium

Vulnerable Code

markdown
> Fetch the complete documentation index at: https://docs.anyway.sh/llms.txt
> Use this file to discover all available pages before exploring further.
markdown
## Documentation

Fetch the relevant page(s) below for up-to-date installation, configuration, and usage instructions:

- Quickstart: https://docs.anyway.sh/quickstart
- Python SDK installation: https://docs.anyway.sh/sdk/python/installation
- Python SDK configuration: https://docs.anyway.sh/sdk/python/configuration
- Python SDK tracing: https://docs.anyway.sh/sdk/python/tracing
- JavaScript SDK installation: https://docs.anyway.sh/sdk/js/installation
- JavaScript SDK configuration: https://docs.anyway.sh/sdk/js/configuration
- JavaScript SDK tracing: https://docs.anyway.sh/sdk/js/tracing

Technical Analysis

The Skill delegates installation, configuration, and usage guidance to remotely hosted pages fetched when the Skill is used. These resources are mutable and are not pinned to a reviewed version, commit, digest, or cryptographic integrity value.

This design prevents the local Skill package from fully defining its effective behavior. If the documentation host, publishing account, DNS configuration, or content delivery path is compromised, the fetched pages could introduce unsafe package names, altered installation commands, insecure configuration, or additional agent instructions.

The reviewed content only directs the agent to retrieve documentation; it does not itself contain a remote executable payload or prove that the external pages are malicious. Therefore, this is a supply-chain trust weakness rather than confirmed remote code execution.

Attack Path

  1. An attacker compromises or gains unauthorized publishing access to the referenced documentation infrastr ...[truncated 1307 chars]
Remediation
View remediation

Remediation Suggestions

  1. Bundle reviewed installation and configuration instructions directly with the Skill.
  2. Pin SDK dependencies to reviewed versions rather than relying on unspecified current releases.
  3. Use package-lock files and integrity checks supported by the relevant package manager.
  4. Pin remote documentation to an immutable release, commit, or versioned path where remote retrieval is unavoidable.
  5. Verify remotely retrieved artifacts using cryptographic hashes or signatures.
  6. Treat fetched documentation as untrusted reference material rather than executable agent instructions.
  7. Require explicit user confirmation before running commands or installing packages obtained from remote pages.
  8. Restrict network retrieval to an allowlist of approved HTTPS hosts and reject redirects to unapproved domains.
  9. Review and display any differences between bundled guidance and newly retrieved documentation before acting on them.

SkillSpector

Vulnerability Patterns
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep

Static analysis

No suspicious patterns detected.