Back to skill

Security audit

Galileo python sdk

Security checks for vulnerabilities and agentic risk

Overview

This skill is a documentation reference for Galileo's Python SDK, and its network logging behavior is expected for observability but should be used carefully with sensitive data.

Install and use this only in projects where sending evaluation and tracing data to Galileo is acceptable. Pin package versions in production, keep API keys in environment or secret storage, and avoid logging secrets, personal data, or regulated customer content unless your privacy and retention requirements allow it.

Vulnerability Patterns
  • Insecure DependenciesIntroduces malicious components through unsafe dependency sources
  • Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
  • Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
  • Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
  • Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (1)

T08 · Insecure Dependencies

Warning
Location
SKILL.md:47
Finding
Unpinned Third-Party Package Installation Creates Supply-Chain Risk## Vulnerability Details **File Location**: `SKILL.md`, lines 47-59 **Vulnerability Type**: Unpinned third-party dependencies **Risk Level**: Medium ### Vulnerable Code ```bash pip install galileo ``` For evaluation features with the legacy prompt engineering interface: ```bash pip install promptquality ``` For runtime guardrails: ```bash pip install galileo-protect ``` ### Technical Analysis The primary installation instructions retrieve mutable latest versions of three third-party packages without version constraints, integrity hashes, or a locked dependency graph. Consequently, the code installed and executed can differ from the version reviewed when this Skill was authored. Python package installation can execute package build logic, while imported packages subsequently execute with the privileges of the consuming application. If a package release, maintainer account, transitive dependency, or configured package index is compromised, following these instructions could introduce attacker-controlled code. Ordinary upstream changes may also alter telemetry behavior or introduce incompatible or vulnerable dependencies. The separately documented `promptquality==1.14.0` constraint in `references/PROMPTQUALITY.md` reduces version drift for that alternative installation path, but the main installation instructions remain unpinned and do not verify package integrity. ### Attack Path 1. An attacker compromises a named package, one of its transitive dependencies, a maintainer publishing account, or the package index used by the environment. 2. The attacker publishes a malicious release that satisfies the unconstrained installation request. 3. A user or agent follows the documented `pip install` command. 4. `pip` resolves and downloads the attacker-controlled release because no reviewed version or hash is required. 5. Malicious build-time code executes during installation, or malicious runtime code executes w ...[truncated 695 chars]
Remediation
## Remediation Suggestions 1. Pin every direct dependency to a reviewed exact version, including `galileo`, `promptquality`, and `galileo-protect`. 2. Generate and maintain a lock file that also constrains transitive dependencies. 3. Record cryptographic hashes and install with `pip --require-hashes` where feasible. 4. Use a trusted or internally mirrored package index and explicitly configure the permitted index source. 5. Scan locked dependencies for known vulnerabilities and review updates before changing pins. 6. Install packages in an isolated virtual environment or container as a non-privileged user. 7. Reconcile the main installation guidance with the documented SDK compatibility matrix so users cannot accidentally install incompatible major versions. 8. Prefer a reviewed requirements file, for example: ```bash python -m pip install --require-hashes -r requirements.txt ``` The requirements file should contain approved exact versions and hashes rather than unconstrained package names.
Vulnerability Patterns
  • Data ExfiltrationExternal Transmission, Env Variable Harvesting, File System Enumeration
  • Privilege EscalationExcessive Permissions, Sudo/Root Execution, Credential Access
  • Prompt InjectionInstruction Override, Hidden Instructions, Exfiltration Commands
  • Supply ChainUnpinned Dependencies, External Script Fetching, Obfuscated Code
  • Excessive AgencyUnrestricted Tool Access, Autonomous Decision Making, Scope Creep
Findings (8)

Credential Access

High
Category
Privilege Escalation
Content
Set the following environment variables:

```bash
# .env file or shell environment
GALILEO_API_KEY="your-api-key"            # Required — from Galileo console
GALILEO_CONSOLE_URL="https://app.galileo.ai"  # Console URL (or self-hosted URL)
GALILEO_PROJECT="my-project"              # Optional — default project
Confidence
60% confidence
Finding
Code accesses credential files (SSH keys, AWS credentials, etc.). This could indicate credential theft attempts.

Missing User Warnings

Medium
Confidence
93% confidence
Finding
The skill prominently promotes automatic tracing, observability, and runtime uploads but does not clearly warn that prompts, model inputs/outputs, retrieved context, tool results, and metadata may be transmitted to Galileo services. In an LLM/observability SDK, this omission can cause developers to enable logging on sensitive workloads and unintentionally exfiltrate proprietary or regulated data.

Missing User Warnings

Medium
Confidence
95% confidence
Finding
The flush example instructs users to upload captured traces to Galileo but gives no adjacent disclosure that those traces may contain sensitive application data collected during prompts, responses, and span logging. Because the example is actionable and easy to copy, users may operationalize data export without understanding the privacy and compliance implications.

Missing User Warnings

Medium
Confidence
93% confidence
Finding
The examples show logging prompts, contexts, queries, and model outputs to Galileo, which implies transmission of potentially sensitive application data to an external service. In SDK reference documentation, omitting any warning about data sensitivity, redaction, or consent can lead developers to copy these patterns into production and unintentionally expose proprietary, personal, or regulated data.

Missing User Warnings

Medium
Confidence
94% confidence
Finding
The documentation states that calls are automatically logged to Galileo with full span details, but it does not warn users that prompts, model inputs/outputs, metadata, or other application data may be transmitted to an external service. In an observability SDK context, this can lead to unintentional disclosure of sensitive data because users may enable tracing without understanding the data-sharing implications.

Missing User Warnings

Medium
Confidence
96% confidence
Finding
The exporter configuration sends OpenTelemetry traces to a remote Galileo endpoint using an API key, but the documentation does not clearly warn that trace payloads may include prompts, responses, tool inputs, and other sensitive telemetry. This omission increases the risk that developers will deploy the example as-is and exfiltrate confidential application or user data to a third-party service unintentionally.

Missing User Warnings

Medium
Confidence
89% confidence
Finding
The examples explicitly show sending inputs, contexts, and model outputs to Galileo for evaluation without any warning that these fields may contain sensitive user data, secrets, or regulated content. In an observability/evaluation SDK, users often paste real prompts and retrieval context into logging pipelines, so omission of privacy guidance can lead to unintended external transmission and retention of confidential data.

Missing User Warnings

Low
Confidence
77% confidence
Finding
The authentication example directs users to connect to `https://app.galileo.ai`, but the markdown does not explain that subsequent evaluation operations may interact with a hosted service and involve remote data processing. For markdown files, omissions of warnings about behaviors affecting privacy should be flagged when the skill description normalizes external service use without disclosure.

Static analysis

No suspicious patterns detected.