{"skill":{"slug":"computer-use","displayName":"Computer Use","summary":"Full desktop computer use for headless Linux servers. Xvfb + XFCE virtual desktop with xdotool automation. 17 actions (click, type, scroll, screenshot, drag,...","description":"---\nname: computer-use\ndescription: Full desktop computer use for headless Linux servers. Xvfb + XFCE virtual desktop with xdotool automation. 17 actions (click, type, scroll, screenshot, drag, etc). Unlike OpenClaw's browser tool, operates at the X11 level so websites cannot detect automation. Includes VNC for live viewing.\nversion: 1.2.1\n---\n\n# Computer Use Skill\n\nFull desktop GUI control for headless Linux servers. Creates a virtual display (Xvfb + XFCE) so you can run and control desktop applications on VPS/cloud instances without a physical monitor.\n\n## Environment\n\n- **Display**: `:99`\n- **Resolution**: 1024x768 (XGA, Anthropic recommended)\n- **Desktop**: XFCE4 (minimal — xfwm4 + panel only)\n\n## Quick Setup\n\nRun the setup script to install everything (systemd services, flicker-free VNC):\n\n```bash\n./scripts/setup-vnc.sh\n```\n\nThis installs:\n- Xvfb virtual display on `:99`\n- Minimal XFCE desktop (xfwm4 + panel, no xfdesktop)\n- x11vnc with stability flags\n- noVNC for browser access\n\nAll services auto-start on boot and auto-restart on crash.\n\n## Actions Reference\n\n| Action | Script | Arguments | Description |\n|--------|--------|-----------|-------------|\n| screenshot | `screenshot.sh` | — | Capture screen → base64 PNG |\n| cursor_position | `cursor_position.sh` | — | Get current mouse X,Y |\n| mouse_move | `mouse_move.sh` | x y | Move mouse to coordinates |\n| left_click | `click.sh` | x y left | Left click at coordinates |\n| right_click | `click.sh` | x y right | Right click |\n| middle_click | `click.sh` | x y middle | Middle click |\n| double_click | `click.sh` | x y double | Double click |\n| triple_click | `click.sh` | x y triple | Triple click (select line) |\n| left_click_drag | `drag.sh` | x1 y1 x2 y2 | Drag from start to end |\n| left_mouse_down | `mouse_down.sh` | — | Press mouse button |\n| left_mouse_up | `mouse_up.sh` | — | Release mouse button |\n| type | `type_text.sh` | \"text\" | Type text (50 char chunks, 12ms delay) |\n| key | `key.sh` | \"combo\" | Press key (Return, ctrl+c, alt+F4) |\n| hold_key | `hold_key.sh` | \"key\" secs | Hold key for duration |\n| scroll | `scroll.sh` | dir amt [x y] | Scroll up/down/left/right |\n| wait | `wait.sh` | seconds | Wait then screenshot |\n| zoom | `zoom.sh` | x1 y1 x2 y2 | Cropped region screenshot |\n\n## Usage Examples\n\n```bash\nexport DISPLAY=:99\n\n# Take screenshot\n./scripts/screenshot.sh\n\n# Click at coordinates\n./scripts/click.sh 512 384 left\n\n# Type text\n./scripts/type_text.sh \"Hello world\"\n\n# Press key combo\n./scripts/key.sh \"ctrl+s\"\n\n# Scroll down\n./scripts/scroll.sh down 5\n```\n\n## Workflow Pattern\n\n1. **Screenshot** — Always start by seeing the screen\n2. **Analyze** — Identify UI elements and coordinates\n3. **Act** — Click, type, scroll\n4. **Screenshot** — Verify result\n5. **Repeat**\n\n## Tips\n\n- Screen is 1024x768, origin (0,0) at top-left\n- Click to focus before typing in text fields\n- Use `ctrl+End` to jump to page bottom in browsers\n- Most actions auto-screenshot after 2 sec delay\n- Long text is chunked (50 chars) with 12ms keystroke delay\n\n## Live Desktop Viewing (VNC)\n\nWatch the desktop in real-time via browser or VNC client.\n\n### Connect via Browser\n\n```bash\n# SSH tunnel (run on your local machine)\nssh -L 6080:localhost:6080 your-server\n\n# Open in browser\nhttp://localhost:6080/vnc.html\n```\n\n### Connect via VNC Client\n\n```bash\n# SSH tunnel\nssh -L 5900:localhost:5900 your-server\n\n# Connect VNC client to localhost:5900\n```\n\n### SSH Config (recommended)\n\nAdd to `~/.ssh/config` for automatic tunneling:\n\n```\nHost your-server\n  HostName your.server.ip\n  User your-user\n  LocalForward 6080 127.0.0.1:6080\n  LocalForward 5900 127.0.0.1:5900\n```\n\nThen just `ssh your-server` and VNC is available.\n\n## System Services\n\n```bash\n# Check status\nsystemctl status xvfb xfce-minimal x11vnc novnc\n\n# Restart if needed\nsudo systemctl restart xvfb xfce-minimal x11vnc novnc\n```\n\n### Service Chain\n\n```\nxvfb → xfce-minimal → x11vnc → novnc\n```\n\n- **xvfb**: Virtual display :99 (1024x768x24)\n- **xfce-minimal**: Watchdog that runs xfwm4+panel, kills xfdesktop\n- **x11vnc**: VNC server with `-noxdamage` for stability\n- **novnc**: WebSocket proxy with heartbeat for connection stability\n\n## Opening Applications\n\n```bash\nexport DISPLAY=:99\n\n# Chrome — only use --no-sandbox if the kernel lacks user namespace support.\n# Check: cat /proc/sys/kernel/unprivileged_userns_clone\n#   1 = sandbox works, do NOT use --no-sandbox\n#   0 = sandbox fails, --no-sandbox required as fallback\n# Using --no-sandbox when unnecessary causes instability and crashes.\nif [ \"$(cat /proc/sys/kernel/unprivileged_userns_clone 2>/dev/null)\" = \"0\" ]; then\n    google-chrome --no-sandbox &\nelse\n    google-chrome &\nfi\n\nxfce4-terminal &                # Terminal\nthunar &                        # File manager\n```\n\n**Note**: Snap browsers (Firefox, Chromium) have sandbox issues on headless servers. Use Chrome `.deb` instead:\n\n```bash\nwget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb\nsudo dpkg -i google-chrome-stable_current_amd64.deb\nsudo apt-get install -f\n```\n\n## Manual Setup\n\nIf you prefer manual setup instead of `setup-vnc.sh`:\n\n```bash\n# Install packages\nsudo apt install -y xvfb xfce4 xfce4-terminal xdotool scrot imagemagick dbus-x11 x11vnc novnc websockify\n\n# Run the setup script (generates systemd services, masks xfdesktop, starts everything)\n./scripts/setup-vnc.sh\n```\n\nIf you prefer fully manual setup, the `setup-vnc.sh` script generates all systemd service files inline -- read it for the exact service definitions.\n\n## Troubleshooting\n\n### VNC shows black screen\n- Check if xfwm4 is running: `pgrep xfwm4`\n- Restart desktop: `sudo systemctl restart xfce-minimal`\n\n### VNC flickering/flashing\n- Ensure xfdesktop is masked (check `/usr/bin/xfdesktop`)\n- xfdesktop causes flicker due to clear→draw cycles on Xvfb\n\n### VNC disconnects frequently\n- Check noVNC has `--heartbeat 30` flag\n- Check x11vnc has `-noxdamage` flag\n\n### x11vnc crashes (SIGSEGV)\n- Add `-noxdamage -noxfixes` flags\n- The DAMAGE extension causes crashes on Xvfb\n\n## Requirements\n\nInstalled by `setup-vnc.sh`:\n```bash\nxvfb xfce4 xfce4-terminal xdotool scrot imagemagick dbus-x11 x11vnc novnc websockify\n```\n","tags":{"latest":"1.2.1"},"stats":{"comments":6,"downloads":15472,"installsAllTime":127,"installsCurrent":126,"stars":40,"versions":5},"createdAt":1769942849740,"updatedAt":1780310954854},"latestVersion":{"version":"1.2.1","createdAt":1771195222594,"changelog":"- Improved description to highlight X11-level automation for undetectable interactions (vs browser tools).\n- Clarified Chrome usage: added instructions to check user namespace support and avoid unnecessary --no-sandbox flag (improves browser stability).\n- Updated manual setup instructions to suggest using the setup script for generating service files.\n- Various documentation refinements for clarity and accuracy.","license":null},"metadata":null,"owner":{"handle":"ram-raghav-s","userId":"s170yk4nq0sd1gdvcavhyj5r5n884wt1","displayName":"Ram-Raghav-S","image":"https://avatars.githubusercontent.com/u/92739966?v=4"},"moderation":{"isSuspicious":false,"isMalwareBlocked":false,"verdict":"clean","reasonCodes":["review.llm_review"],"summary":"Review: review.llm_review","engineVersion":"v2.4.24","updatedAt":1779932779903}}