Security checks for vulnerabilities and agentic risk
Overview
This is a coherent inventory-planning skill, but users should treat its order, markdown, and inventory-change advice as decision support requiring normal business approval.
Installers should treat this as a planning reference, not as authority to make live business changes. Require explicit human approval before creating purchase orders, canceling POs, changing WMS/POS inventory records, executing markdowns, liquidation, public statements, or vendor communications. Avoid running artifact/evals/run_evals.py unless the surrounding repository and imported shared framework path are trusted.
Insecure Skill Coding PracticesFinds exploitable flaws such as hardcoded secrets or command injection
Skill Instruction HijackingAlters the agent's session goals or safety constraints when the skill loads
Agent Memory PoisoningWrites attacker-controlled rules into memory that affect later sessions
Remote Payload Retrieval and ExecutionFetches external code whose behavior can change after review
Embedded Malicious CodeShips malicious scripts inside the skill and executes them locally
Findings (1)
T09 · Insecure Skill Coding Practices
Warning
Location
evals/run_evals.py:7
Finding
Python Path Injection Enables Arbitrary Code Execution from an Untrusted Temporary Directory
Content
View full analysis
Vulnerability Details
File Location: evals/run_evals.py, lines 7–10 Vulnerability Type: Python module search-path injection from an external, writable directory Risk Level: Medium
Vulnerable Code:
python
REPO_ROOT = Path(__file__).resolve().parent.parent.parent.parent
sys.path.insert(0, str(REPO_ROOT / "shared"))
from eval_framework import main
Technical Analysis
The script derives REPO_ROOT by traversing four parent directories from evals/run_evals.py. At the audited project location, this resolves to /tmp, causing /tmp/shared to be inserted at index zero of sys.path.
Python searches sys.path in order when resolving imports. Consequently, from eval_framework import main loads an eval_framework.py file or eval_framework package from /tmp/shared before checking lower-priority trusted locations. Because this path is outside the audited project and is located beneath a predictable temporary directory, another local user or process may be able to populate it with attacker-controlled Python code.
Importing a Python module executes its top-level statements immediately. The attacker therefore does not need to provide a valid evaluator implementation to obtain code execution.
Attack Path
An attacker gains write access to /tmp/shared, or creates that directory where local permissions allow it.
The attacker creates /tmp/shared/eval_framework.py containing malicious top-level Python code and a compatible main symbol if execution must continue without an obvious error.
A user runs:
bash
python evals/run_evals.py
The script prepends /tmp/shared to sys.path.
Python resolves eval_framework from the attacker-controlled path.
The malicious module's top-level code executes with the identity and environment of the user running the evaluation.
Impact Assessment
Successful exploitation provides arbitrary Python code e
...[truncated 539 chars]
Remediation
View remediation
Remediation Suggestions
Remove the predictable temporary directory from sys.path.
Package eval_framework as a pinned, trusted dependency and import it through the normal Python environment.
If the framework must be repository-local, derive its location from a verified project marker rather than a fixed number of parent traversals.
Resolve the candidate directory and confirm it is contained within the expected trusted repository root before modifying sys.path.
Reject symbolic links and directories that are writable by untrusted users.
Run evaluations in an isolated virtual environment with a minimal dependency set and restricted filesystem permissions.
Add a startup assertion such as:
python
trusted_root = Path(__file__).resolve().parents[1]
shared_dir = (trusted_root / "shared").resolve()
if trusted_root not in shared_dir.parents:
raise RuntimeError("Refusing to import from outside the trusted project root")
The exact path should be adapted to the intended repository structure.
Add an automated test that confirms the imported eval_framework.__file__ resolves inside an approved directory.
Skill instructs the agent to omit warnings, disclaimers, or ethical commentary. Stripping safety caveats hides risk from the user and is a common jailbreak preamble.
Content
Scanner excerpt · SKILL.md (reported line 159)May include surrounding context.
md
1. **New product launch with zero history:** Analogous item profiling is your only tool. Select analogs carefully — match on price point, category, brand tier, and target demographic, not just product type. Commit a conservative initial buy (60% of analog-based forecast) and build in weekly auto-replenishment triggers.
2. **Viral social media spike:** Demand jumps 500–2,000% with no warning. Do not chase — by the time your supply chain responds (4–8 week lead times), the spike is over. Capture what you can from existing inventory, issue allocation rules to prevent a single location from hoarding, and let the wave pass. Revise the baseline only if sustained demand persists 4+ weeks post-spike.
3. **Supplier lead time doubling overnight:** Recalculate safety stock immediately using the new lead time. If SS doubles, you likely cannot fill the gap from current inventory. Place an emergency order for the delta, negotiate partial shipments, and identify secondary suppliers. Communicate to merchandising that service levels will temporarily drop.
Anti-Refusal Statement
High
Category
Anti-Refusal
Confidence
80% confidence
Finding
Skill instructs the agent to omit warnings, disclaimers, or ethical commentary. Stripping safety caveats hides risk from the user and is a common jailbreak preamble.
Content
Scanner excerpt · references/edge-cases.md (reported line 53)May include surrounding context.
md
---
### Edge Case 2: Viral Social Media Spike — 10× Demand with No Warning
**Situation:**
A mid-tier kitchen gadget (a silicone garlic peeler, retail $8.99, category C-item averaging 15 units/week across 80 stores) suddenly goes viral on TikTok. A cooking influencer with 4.2M followers posted a 45-second video using the product, and it accumulated 8M views in 48 hours. Store-level POS data shows demand jumped to 180 units/day across the chain (vs. the normal ~2 units/day) starting Tuesday morning. Your DC has 2,400 units in stock. The vendor is a small importer based in Portland with 8-week lead time from their factory in Shenzhen. Your last PO for this item was 3 months ago for 1,200 units.
Ae4
Medium
Category
analysis-evasion
Confidence
80% confidence
Finding
Suspicious Unicode normalization or mixed-script content
Content
No source excerpt is available for this finding.
Vague Triggers
Medium
Category
Not specified by scanner
Confidence
95% confidence
Finding
The manifest uses broad activation language such as 'Use when forecasting demand, setting safety stock, planning replenishment, managing promotions, or optimising inventory levels,' which can cause the skill to be invoked for a wide range of generic planning requests. Over-broad routing increases the chance this skill overrides more appropriate domain, compliance, or human-review workflows and can lead to unsafe or low-context business decisions being treated as authoritative.
Content
No source excerpt is available for this finding.
Ae4
Medium
Category
analysis-evasion
Confidence
80% confidence
Finding
Suspicious Unicode normalization or mixed-script content
Content
No source excerpt is available for this finding.
Natural-Language Policy Violations
Medium
Category
Not specified by scanner
Confidence
84% confidence
Finding
The embedded agent response begins with "I'll solve this step-by-step" and all scenario responses throughout the JSON are written exclusively in English, with no indication that language is user-selectable. Because SQP-3 applies to all file types and flags language/locale policy violations, this is a policy concern if these strings are used as fixed skill outputs or exemplars without opt-in.
Content
No source excerpt is available for this finding.
Ae4
Medium
Category
analysis-evasion
Confidence
80% confidence
Finding
Suspicious Unicode normalization or mixed-script content
Content
No source excerpt is available for this finding.
Ae4
Medium
Category
analysis-evasion
Confidence
80% confidence
Finding
Suspicious Unicode normalization or mixed-script content
Content
No source excerpt is available for this finding.
Ae4
Medium
Category
analysis-evasion
Confidence
80% confidence
Finding
Suspicious Unicode normalization or mixed-script content
Content
No source excerpt is available for this finding.
Autonomous Decision Making
Medium
Category
Excessive Agency
Confidence
75% confidence
Finding
Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.
Content
Scanner excerpt · evals/scenarios.json (reported line 1255)May include surrounding context.
json
"weight": 0.25,
"rubric": {
"pass": "Verifies: S = demand during (LT + R) + SS. Demand during (2+1) = 85 × 3 = 255 units. SS = 1.65 × 18 × √3 ≈ 51 units. S = 255 + 51 = 306 units. Notes the system's 350 target is somewhat higher than the calculated 306 — this could reflect a buffer or a rounding up to case packs. Either accepts the 350 as reasonable or flags the 44-unit discrepancy for investigation.",
"fail": "Does not verify the calculation. Accepts 350 without checking. Gets the formula fundamentally wrong."
}
},
{
Ae4
Medium
Category
analysis-evasion
Confidence
80% confidence
Finding
Suspicious Unicode normalization or mixed-script content
Content
No source excerpt is available for this finding.
Ae4
Medium
Category
analysis-evasion
Confidence
80% confidence
Finding
Suspicious Unicode normalization or mixed-script content
Content
No source excerpt is available for this finding.
Autonomous Decision Making
Medium
Category
Excessive Agency
Confidence
75% confidence
Finding
Skill enables autonomous high-impact decisions without human-in-the-loop verification. Critical operations (destructive commands, financial transactions, data deletion) should require explicit user confirmation.
Content
Scanner excerpt · references/decision-frameworks.md (reported line 744)May include surrounding context.
md
1. **All overrides must be documented** with a reason code and quantitative justification.
2. **Override authority by magnitude:**
- ±10%: Planner can override without approval
- ±10–25%: Requires planning manager approval
- ±25–50%: Requires director approval
- > ±50%: Requires VP approval (or planning committee)
Ae4
Medium
Category
analysis-evasion
Confidence
80% confidence
Finding
Suspicious Unicode normalization or mixed-script content
Content
No source excerpt is available for this finding.
Natural-Language Policy Violations
Low
Category
Not specified by scanner
Confidence
81% confidence
Finding
The text states that all thresholds, formulas, and cost assumptions reflect US multi-location retail operations, which is a locale-specific constraint expressed as a blanket default. Because the document does not offer an opt-in choice or alternative locale handling, this can conflict with policies against forcing a specific locale unless explicitly justified for the user's context.
Content
No source excerpt is available for this finding.
Scope Creep
Low
Category
Excessive Agency
Confidence
70% confidence
Finding
Skill's behavior or capabilities extend beyond its stated purpose. Scope creep allows an agent to perform actions unrelated to its documented functionality, increasing the attack surface.
Content
Scanner excerpt · references/edge-cases.md (reported line 255)May include surrounding context.
md
Your highest-velocity SKU in the beverage category (a 24-pack water case, A-item, ~1,200 units/week across 80 stores) has been showing 95%+ in-stock rate in the system, but customer complaints about out-of-stocks have tripled in the past month. The WMS shows 3,400 units at the DC and the stores collectively show 2,100 units on hand. However, three separate stores have reported that they can't find the product despite the system showing 50–80 units each. A partial cycle count at the DC reveals an actual count of 2,100 units — the WMS is overstating by 1,300 units (38% phantom inventory).
**Why It's Tricky:**
Every replenishment decision for the past several weeks has been based on a position that was 1,300 units higher than reality. The system thinks the DC has 4.7 weeks of supply when it actually has 2.9 weeks. Stores are running out because store-level inventory is also likely overstated (if receiving errors or shrinkage are the cause). The problem is almost certainly not limited to this one SKU — whatever process caused the phantom inventory (receiving errors, system timing, shrinkage) is likely affecting other items.
**Common Mistake:**
Correcting the inventory in the WMS and moving on. The correction fixes the symptom but not the cause. Next month, phantom inventory will reappear.
