T09 · Insecure Skill Coding Practices
- Location
verify/eval_toolkit_quality_test.py:62- Finding
Predictable Shared Temporary File Permits Symlink-Based File Creation Redirection
- Content
View full analysis
Vulnerability Details
File Location:
verify/eval_toolkit_quality_test.py, lines 62–63; related file-creation logic intools/eval_toolkit.py, lines 26–31
Vulnerability Type: Predictable temporary file and time-of-check/time-of-use weakness
Risk Level: LowVulnerable Code
python rc, out = run_tool([ "dataset", "--action", "init", "--out", "/tmp/evalset_t.jsonl" ])The invoked implementation uses a separate existence check followed by a symlink-following write:
python def dataset_init(out): if os.path.exists(out): print(f"Error: {out} already exists (not overwritten).") return 2 with open(out, "w", encoding="utf-8") as f: f.write(DATASET_TEMPLATE + "\n")Technical Analysis
The verification script uses the globally predictable path
/tmp/evalset_t.jsonl. Shared temporary directories are normally writable by other local users, so an attacker can prepare this path before the verifier runs.The target creation logic first calls
os.path.exists()and later callsopen()in write mode. These are separate filesystem operations, creating a time-of-check/time-of-use window. Furthermore,open()follows symbolic links.A pre-existing regular file or symlink to an existing file causes the test to fail because
os.path.exists()returns true. More importantly, a dangling symbolic link can pass the existence check becauseos.path.exists()returns false when the link target does not exist. The subsequentopen()follows that link and creates the fixed dataset template at the attacker-selected destination, provided the victim has permission to create that destination.Attack Path
- A local attacker determines that the verifier uses
/tmp/evalset_t.jsonl. - The attacker creates a dangling symbolic link at that path whose target is another nonexistent path writable or creatable by the victim:
bash ln -s /victim-writable-directory/attacker-selected-file /tmp/evalset_t.j
...[truncated 1055 chars]
- A local attacker determines that the verifier uses
- Remediation
View remediation
Remediation Suggestions
-
Replace the fixed shared path with a securely generated private temporary directory:
python import tempfile with tempfile.TemporaryDirectory(prefix="eval-toolkit-test-") as temp_dir: output_path = os.path.join(temp_dir, "evalset.jsonl") rc, out = run_tool([ "dataset", "--action", "init", "--out", output_path, ]) -
Make dataset creation atomic by replacing the existence-check/write sequence with exclusive creation:
python def dataset_init(out): try: with open(out, "x", encoding="utf-8") as f: f.write(DATASET_TEMPLATE + "\n") except FileExistsError: print(f"Error: {out} already exists (not overwritten).") return 2 return 0 -
Where supported, harden low-level creation against symbolic-link traversal by using
os.open()withos.O_CREAT | os.O_EXCLandos.O_NOFOLLOW, then wrap the descriptor withos.fdopen(). -
Ensure test artifacts are removed automatically, including when a test fails or times out.
-
Avoid hardcoded names in globally writable directories in all future verification cases.
-