T06 · System Persistence
Error
- Location
- scripts/remote_train.py:480
- Finding
- Persistent Passwordless SSH Access Installed on Remote Systems<![CDATA[ ## Vulnerability Details **File Location**: `scripts/remote_train.py:480-520`; related workflow in `references/remote_training.md:106-118` **Vulnerability Type**: Persistent SSH authorization **Risk Level**: Critical ### Vulnerable Code ```python # 1. Generate key pair (no passphrase) if os.path.exists(DEFAULT_KEY_PATH): print(f"Key already exists: {DEFAULT_KEY_PATH}") overwrite = input("Overwrite? (y/N): ").strip().lower() if overwrite != 'y': print("Cancelled") sys.exit(1) print(f"Generating key pair...") keygen_cmd = [ "ssh-keygen", "-t", "ed25519", "-f", DEFAULT_KEY_PATH, "-N", "", "-C", "deepspeed-remote" ] result = subprocess.run(keygen_cmd, capture_output=True, text=True) if result.returncode != 0: print(f"Key generation failed: {result.stderr}") sys.exit(1) os.chmod(DEFAULT_KEY_PATH, 0o600) os.chmod(f"{DEFAULT_KEY_PATH}.pub", 0o644) # 2. Read public key pub_key_path = f"{DEFAULT_KEY_PATH}.pub" with open(pub_key_path) as f: pub_key = f.read().strip() # 3. Auto-upload public key via SSH (password available) pub_key_b64 = base64.b64encode(pub_key.encode()).decode() remote_setup = ( f"mkdir -p ~/.ssh && " f"chmod 700 ~/.ssh && " f"touch ~/.ssh/authorized_keys && " f"chmod 600 ~/.ssh/authorized_keys && " f"grep -qF '{pub_key_b64}' ~/.ssh/.deepspeed_setup_marker 2>/dev/null || " f"(echo '{pub_key_b64}' >> ~/.ssh/.deepspeed_setup_marker && " f"echo '{pub_key_b64}' | base64 -d >> ~/.ssh/authorized_keys)" ) ``` ### Technical Analysis The `setup-keys` workflow generates an SSH private key with an empty passphrase and appends its public key to the remote account's `~/.ssh/authorized_keys`. This creates an authorization mechanism that remains valid after the training process and SSH ControlMaster session have ended. The operation is related to remote training, but it exceeds the minimum access duration required to launch or monitor an individual training job. The `s ...[truncated 1434 chars]
- Remediation
- <![CDATA[ ## Remediation Suggestions 1. Require explicit, informed confirmation immediately before modifying `authorized_keys`. 2. Clearly state that the key grants continuing login access after training ends. 3. Generate a unique, task-scoped key for each remote job rather than reusing a global key. 4. Revoke the public key automatically when training completes, is stopped, or expires. 5. Securely delete the corresponding local task key after revocation. 6. Add restrictive authorized-key options where possible, such as: - `from="<trusted-source-address>"` - `command="<restricted-training-wrapper>"` - `no-agent-forwarding` - `no-port-forwarding` - `no-X11-forwarding` - `no-pty` 7. Prefer an existing user-managed SSH agent or credential rather than creating a new persistent identity. 8. Add a dedicated cleanup command that reliably removes both the public key and `.deepspeed_setup_marker`. 9. Avoid using a privileged remote account when a dedicated, restricted training account is sufficient. ]]>