agent-android

Control Android over LAN without USB, ADB, or root.

Security Scan

The skill's requirements and runtime instructions match its stated purpose (LAN control of an Android device) and request no unrelated credentials, but you should verify the CLI binary's provenance before installing and be cautious with screenshots/UI dumps.

Purpose & Capability

Name/description, declared required binary (agent-android), and the commands in SKILL.md all align with a LAN Android control CLI. No unrelated credentials, binaries, or config paths are requested.

Instruction Scope

SKILL.md limits actions to interacting with a user-provided device URL and outlines specific commands (health, apps, tap, input, screenshot, etc.). It explicitly warns not to scan networks and to treat UI trees/screenshots as sensitive — instructions do not reference unrelated files, env vars, or external endpoints.

Install Mechanism

The install spec uses 'uv' to install package 'aivane-agent-android' which creates the agent-android binary. 'uv' is not a widely-known global package host like official GitHub releases/homebrew; installing a third-party CLI writes a binary to disk and should be verified. This is not necessarily malicious but is a provenance/trust risk to check.

Credentials

The skill requests no environment variables or credentials. The only sensitive I/O is UI trees and screenshots returned by the device API (expected for this purpose); SKILL.md instructs to save such data only on explicit user request.

Persistence & Privilege

always:false and default autonomous invocation are set; the skill does not request permanent inclusion or modify other skills. Installation creates a single CLI binary which is expected for this use-case.

Assessment

This skill appears coherent for controlling an Android device over LAN. Before installing: (1) verify the aivane-agent-android/agent-android binary provenance (check the GitHub repo, release artifacts, and package contents) because the 'uv' install mechanism is less familiar; (2) only give the skill explicit, trusted device URLs on your LAN — do not let it discover or scan your network; (3) treat UI trees and screenshots as sensitive and avoid saving or exposing them unless you explicitly request a file; (4) consider running the CLI in an isolated environment or inspecting its binary/script contents after install; (5) ensure the user confirms any actions that affect private apps, accounts, or irreversible operations. If you want higher assurance, request the package source code or a signed release artifact before installing.

Android REPL

Use this skill to drive an Android device through the public agent-android beta surface published in the aivanelabs/ai-rpa repo.

Runtime prerequisites:

• agent-android is available on PATH
• if it is missing, install the CLI with uv tool install aivane-agent-android; then run uv tool update-shell if the command is still not found
• the user has provided a trusted device URL such as http://<device-ip>:8080

The public path is local-first:

• the phone hosts the lightweight HTTP service locally
• the desktop connects directly to http://<device-ip>:8080
• UI reads, taps, inputs, and screenshots stay on the phone and controlling machine
• the current tradeoff is LAN-only control; an optional server-side or relay path may arrive later

If an agent-android command suddenly stops working, first check whether the AIVane app or its local API service has exited on the phone.

Safety Boundaries

• Connect only to a device URL explicitly provided by the user.
• Do not scan local networks or guess device IP addresses.
• Treat UI trees and screenshots as sensitive because they may contain private app content.
• Save screenshots or raw UI dumps only when the user explicitly asks for a file output.
• Ask for confirmation before operating sensitive apps, private content, account settings, or irreversible actions.
• Do not expose the phone-side service on a public network.

Core Workflow

Every Android control task should follow the same short loop:

1. Confirm connectivity with /health
2. Discover the target app with apps if needed
3. Launch one app
4. Inspect the current UI tree
5. Perform one action
6. Inspect again

Keep the loop short. Prefer inspect -> act -> inspect over long speculative command chains.

Quick Start

Start the REPL with the user-provided device URL:

• agent-android --repl --url http://<device-ip>:8080

Built-in help:

• agent-android --help
• In the REPL: h

Essential CLI Commands

Use the one-off CLI when you already know the exact action you want.

Connectivity:

• agent-android --health --url http://<device-ip>:8080

Discovery:

• agent-android --apps --url http://<device-ip>:8080
• agent-android --list --url http://<device-ip>:8080
• agent-android --id com.example:id/search --url http://<device-ip>:8080
• agent-android --text Search --url http://<device-ip>:8080
• agent-android --inputs --url http://<device-ip>:8080
• agent-android --refId 7 --url http://<device-ip>:8080
• agent-android --xpath 7 --url http://<device-ip>:8080
• agent-android --get-attr 7 text --url http://<device-ip>:8080

Actions:

• agent-android --launch com.xingin.xhs --url http://<device-ip>:8080
• agent-android --tap 7 --url http://<device-ip>:8080
• agent-android --input 7 "hello world" --url http://<device-ip>:8080
• agent-android --swipe up --url http://<device-ip>:8080
• agent-android --back --url http://<device-ip>:8080
• agent-android --press home --url http://<device-ip>:8080
• agent-android --screenshot --url http://<device-ip>:8080

Waiting and output:

• agent-android --wait-for Search --timeout 30 --url http://<device-ip>:8080
• agent-android --list --raw --url http://<device-ip>:8080

REPL Command Reference

Use the REPL for exploratory tasks and smoke runs. Short aliases and long names both work.

Browse

• health or hl Check /health.
• l [n] or list [n] List the first n elements, or all cached elements when n is omitted.
• ss or snapshot Force-refresh the UI tree and print it again.
• f <text> or find <text> Filter by text or content description.
• id <resourceId> Filter by Android resource ID.
• ref <refId> Show the full element detail for one refId.
• x <refId> or xpath <refId> Generate XPath candidates and validate their runtime match counts.
• xx <refId> Tap by the best unique generated XPath candidate. Refuses ambiguous matches.
• vx <xpath> [idx] or validatex <xpath> [idx] Validate one XPath against the runtime. Optionally inspect one match by zero-based index.

Interact

• t <refId> or tap <refId> Tap the element center point from the current tree.
• tx <xpath> or tapx <xpath> Tap one runtime-resolved XPath target.
• i <refId> <text> or input <refId> <text> Input text into a refId target. Use --clear or "" to clear instead of typing.
• ix <xpath> <text> or inputx <xpath> <text> Input text into one XPath target. Use ix <xpath> -- or --clear to clear the field.
• sw <d|u|l|r> [--dur N] [--dist N] or swipe ... Swipe down/up/left/right with optional duration and distance.
• p <back|home|recents> or press ... Press a system key.
• b or back Press Back.
• la <package> or launch <package> Launch an app by package name.
• s [path] or screenshot [path] Capture a screenshot to an auto-generated or explicit path.

Wait And Inspect

• wf <text> [--t N] or waitfor ... Wait for an element to appear.
• g <refId> <attr> or get <refId> <attr> Read an attribute such as text, class, bounds, x, y, or xpath.
• apps List launcher apps from /apps.

Session

• raw Toggle raw JSON mode.
• vars Show current URL, timeout, raw mode, and tree cache state.
• set timeout 30 Set the default wait timeout in seconds.
• h or help Show the built-in help text.
• q or quit Exit the REPL.

Common Patterns

First smoke flow

1. Start the REPL with agent-android --repl --url http://<device-ip>:8080.
2. health
3. apps
4. la <package>
5. l
6. t <refId>
7. i <refId> hello
8. b
9. s

Find the right package before launch

• apps
• la com.example.app
• l

Inspect an element before using XPath

• l
• ref 12
• x 12
• vx //EditText[@text='Search']

Clear and refill an input

• i 7 --clear
• i 7 hello world
• ix //EditText[@text='Search'] --
• ix //EditText[@text='Search'] -- hello world

Troubleshooting

• If an agent-android command fails, first check whether the AIVane app or phone-side API service has exited.
• If agent-android is not found, run uv tool update-shell, reopen the terminal, and retry.
• Re-open the app or restart the phone-side service, then retry curl http://<device-ip>:8080/health.
• If health works but UI commands fail, run ss to force-refresh the tree before tapping or inputting.
• If tx or ix fails, run vx <xpath> and make sure the XPath resolves to exactly one runtime match.
• If screenshots fail the first time, confirm the on-device MediaProjection permission prompt was accepted.
• If everything suddenly stops responding, confirm the phone IP did not change and that the desktop is still on the same LAN.

When To Stop

Stop and ask for user help when:

• the device is unreachable on LAN
• the app is not running and cannot be restarted from the current path
• required Android permissions are missing
• launcher discovery returns nothing useful
• the runtime UI no longer matches the expected screen after repeated refreshes

References

• Smoke checklist: GitHub
• Quickstart: GitHub
• Install guide: GitHub
• Public protocol: GitHub
• Known beta limits: GitHub