1ai Autodroid

Automation

Control and automate Android devices via ADB, vision AI, or accessibility bridge for app interaction, multitasking, screenshots, messaging, and flows.

Install

openclaw skills install 1ai-autodroid

— If this skill saves you time, consider tipping: https://www.tip.md/oyi77

1ai-autodroid — Ultimate Android Control

Decision Tree: Which Mode to Use

Device connected?
├── USB + ADB authorized? → Use ADB Mode (fastest, no setup)
│   ├── Need structured UI? → uiautomator dump
│   ├── Need visual? → screencap → image tool → tap
│   └── Need reliable text-click? → uiautomator2 Python
└── WiFi only (no ADB)? → Use cua HTTP Mode (requires ClawPaw APK)
    ├── Multi-device → cua -d <name> <cmd>
    └── Flow engine → cua flow '{steps}' (100ms poll, no LLM)

Mode 1: ADB Direct (USB, No Dependencies)

ADB=~/.local/bin/adb  # or just: adb

# Status
$ADB devices -l
python3 skills/1ai-autodroid/scripts/autodroid.py status

# Screenshot → AI analyze
python3 skills/1ai-autodroid/scripts/autodroid.py screenshot
# → /tmp/autodroid_screen.png → pass to image tool

# Actions
python3 skills/1ai-autodroid/scripts/autodroid.py tap 360 820
python3 skills/1ai-autodroid/scripts/autodroid.py swipe 360 1200 360 400 300
python3 skills/1ai-autodroid/scripts/autodroid.py type "hello world"
python3 skills/1ai-autodroid/scripts/autodroid.py key HOME|BACK|POWER|ENTER
python3 skills/1ai-autodroid/scripts/autodroid.py launch com.bca
python3 skills/1ai-autodroid/scripts/autodroid.py unlock [PIN]

UIAutomator Dump (Find Element Coordinates)

$ADB shell uiautomator dump /sdcard/ui.xml && $ADB pull /sdcard/ui.xml /tmp/ui.xml
grep -o 'text="[^"]*" .*bounds="[^"]*"' /tmp/ui.xml | head -30
# → finds bounds="[x1,y1][x2,y2]" → tap center: ((x1+x2)/2, (y1+y2)/2)

Root Fallback Pattern

# Try non-root first, fallback to root
input tap 540 1600 2>/dev/null || su -c "input tap 540 1600"
screencap -p /sdcard/s.png 2>/dev/null || su -c "screencap -p /sdcard/s.png"

Mode 2: Vision AI (No Coordinates Needed)

When coordinates are unknown:

python3 skills/1ai-autodroid/scripts/autodroid.py screenshot
Pass /tmp/autodroid_screen.png to image tool with prompt:

"Find [element description]. Return its center as X,Y coordinates."
Parse X,Y from response
python3 skills/1ai-autodroid/scripts/autodroid.py tap X Y

Mode 3: uiautomator2 Python (Text-Based, Reliable)

import uiautomator2 as u2  # pip install uiautomator2 -q (first time)
d = u2.connect()            # auto-detects USB device

d.app_start("com.bca")
d(text="Login").click()
d(resourceId="com.bca:id/password").set_text("mypass")
d(text="Masuk").click()

# By description
d(description="Search").click()
# Screenshot
d.screenshot("/tmp/u2_screen.png")

First-time setup:

pip install uiautomator2 -q
python3 -m uiautomator2 init  # installs ATX agent on device

Mode 4: cua HTTP Bridge (ClawPaw APK, Most Powerful)

Requires ClawPaw APK installed on device with Accessibility Service enabled. See references/cua-setup.md for install guide.

# Device management
cua add phone1 192.168.x.x <token>
cua devices
cua use phone1

# Perception
cua screen -c        # compact a11y tree (fastest for AI decisions)
cua screenshot       # visual screenshot
cua notifications    # read all notifications
cua info             # model, screen size, permissions

# Actions (text-based = resolution-independent)
cua click "Login"    # tap by visible text ← PREFERRED over coordinates
cua tap 540 1200     # tap by coordinates (fallback)
cua type "hello"     # type text (CJK supported)
cua swipe up|down|left|right
cua back && cua home
cua launch com.bca

# Device I/O
cua sms list 10         # read SMS
cua sms send +62xxx "msg" # send SMS
cua contacts "John"     # search contacts
cua camera front        # take selfie → returns path
cua location            # GPS location
cua clipboard           # read clipboard
cua clipboard "text"    # write clipboard
cua tts "hello"         # speak via phone speaker
cua battery             # battery status
cua wifi                # WiFi info

# Unlock
cua config pin 123456   # store PIN once
cua unlock              # auto-unlock anytime

# Flow Engine (100ms poll, ZERO LLM calls per step)
cua flow '{
  "steps": [
    {"wait": "Login", "then": "tap", "timeout": 10000},
    {"wait": "Masuk", "then": "tap", "timeout": 5000},
    {"wait": "Beranda", "then": "none", "timeout": 15000}
  ]
}'

# Multi-device
cua -d phone1 screenshot
cua -d tablet screen -c

Rule: Prefer cua click "text" over cua tap X Y — text-based is reliable across screen sizes.

Common App Packages

App	Package
BCA Mobile	`com.bca`
WhatsApp	`com.whatsapp`
TikTok	`com.zhiliaoapp.musically`
Instagram	`com.instagram.android`
Shopee	`com.shopee.id`
Tokopedia	`com.tokopedia.tkpd`
Gojek/GoPay	`com.gojek.app`
OVO	`ovo.id`
Dana	`com.dana`
Telegram	`org.telegram.messenger`
YouTube	`com.google.android.youtube`
Chrome	`com.android.chrome`
Settings	`com.android.settings`

Ready-Made Flows

See references/flows.md for:

BCA Balance Check (ADB + vision)
WhatsApp Send Message
TikTok App Launch + Record
Screen Unlock
Vision Tap pattern

Safety Rules

Financial actions (transfer, payment) → ALWAYS confirm with user before executing
SMS send → confirm recipient + content
Never hardcode credentials → read from env or prompt
Screen lock → check and unlock first before automating
Coordinates vs text → prefer text-based click when available (resolution-independent)
Wait between actions → min 500ms between steps for UI to settle

Troubleshooting

Problem	Fix
`unauthorized`	Enable USB debugging, accept on device
`offline`	`adb kill-server && adb start-server`
Screenshot black	Screen locked → unlock first
Tap wrong position	Screenshot scaled ≠ actual — use uiautomator bounds
cua not found	Install ClawPaw APK, see `references/cua-setup.md`
App crashes after launch	Wait longer: `time.sleep(3)`