MacOS Desktop Control (Mouse, Keyboard, Screenshots)

Automate macOS desktop by capturing screenshots and executing precise mouse movements, clicks, and keyboard inputs via cliclick.

Audits

Suspicious

ClawScanSuspicious

Agentic behavior and permission review.

Static analysisPass

Pattern checks against bundled files.

VirusTotalPass

Multi-engine malware detections and file reputation.

Install

openclaw skills install macos-desktop-control

macOS Control Skill

A high-fidelity automation bridge for macOS (Darwin) that enables agents to perceive the desktop state and execute precise mouse and keyboard interactions.

🛠 Included Scripts

This skill leverages specialized wrappers located in the /scripts directory to interface with system-level binaries:

1. `cliclick_wrapper.sh`

A dedicated wrapper for the cliclick utility to handle synthetic input events.

Path: scripts/cliclick_wrapper.sh
Logic: Executes /opt/homebrew/bin/cliclick with passed arguments.
Capabilities: Left/Right clicks, mouse movement, and keyboard emulation.

2. `vision_wrapper.sh`

The visual perception engine for the skill.

Path: scripts/vision_wrapper.sh
Logic: Utilizes the native macOS screencapture utility in silent mode (-x).
Output: Generates a standard PNG at /tmp/claw_view.png.

🚀 Tool Specifications

`see`

Captures the current screen state for visual analysis.

Returns: A string confirming the filepath of the capture.
Use Case: Identifies UI elements, window positions, and application states.

`click`

Sends precise commands to the mouse and keyboard.

Usage: click "c:x,y" (Click) or click "m:x,y" (Move).
Syntax: Supports all cliclick standard notation including w: (wait) and t: (type).

⚙️ Requirements & Setup

Binary Dependency:
```
brew install cliclick
```