T08 · Insecure Dependencies
- Location
evals/README.md:75- Finding
Unpinned Third-Party SDK Installation
- Content
View full analysis
Vulnerability Details
File Location:
evals/README.md:75
Vulnerability Type: Unpinned third-party dependency installation
Risk Level: MediumVulnerable Code Snippet:
text (`pip install anthropic`) and set `ANTHROPIC_API_KEY`. `--dry-run` needs neither and prints the plan, which is enough to catch a broken runner in CI.Technical Analysis
The evaluation documentation instructs users to install the
anthropicpackage directly from the package index without a version constraint, lockfile, integrity hash, or isolated-environment requirement. Consequently, installation is not reproducible and automatically resolves whichever release is current when the command is run.The package name is legitimate and no evidence shows that its current releases are malicious. The risk is conditional: if the upstream package, its distribution account, or its dependency chain were compromised, following the documented command could install attacker-controlled code. An unexpected incompatible release could also alter evaluation behavior.
Python packages and their transitive dependencies execute with the privileges of the invoking user during installation or subsequent import.
evals/run_scenarios.pyimports this dependency and creates an API client, making installed package code part of the trusted execution path.Attack Path
- An attacker compromises the upstream package publication account, release infrastructure, or a transitive dependency.
- The attacker publishes a malicious release under a version accepted by the unrestricted installation command.
- A user follows the documented
pip install anthropicinstruction. - Pip resolves and installs the malicious or compromised release.
- Attacker-controlled code executes during package installation or when
evals/run_scenarios.pyimports the package. - That code operates with the permissions of the user running pip or the evaluation harness.
...[truncated 684 chars]
- Remediation
View remediation
Remediation Suggestions
- Pin the reviewed direct dependency to an exact version in a dedicated evaluation requirements file.
- Include hashes for the package and all transitive dependencies, then install with hash enforcement:
bash python3 -m pip install --require-hashes -r requirements-evals.txt - Generate and review a lockfile using a dependency-locking tool, and update it through controlled dependency-review changes.
- Recommend installation in a dedicated virtual environment:
bash python3 -m venv .venv .venv/bin/python -m pip install --require-hashes -r requirements-evals.txt - Run dependency vulnerability and provenance checks in CI.
- Avoid running installation or evaluation under an administrator account.
- Limit the API key used for evaluations where provider-side controls permit, and keep unrelated secrets out of the evaluation environment.
