Macos Gui Automation
WarnAudited by ClawScan on May 10, 2026.
Overview
The skill is transparent about controlling and reading your Mac’s GUI, but that access is broad enough that it should be reviewed and used only with supervision.
Install only if you intentionally want an agent to read and control your Mac’s GUI. Keep sessions supervised, avoid displaying secrets, require explicit confirmation for impactful actions, install dependencies from trusted sources, and revoke macOS Accessibility/Screen Recording permissions when you no longer need them.
Findings (4)
Artifact-based informational review of SKILL.md, metadata, install specs, static scan signals, and capability signals. ClawScan does not execute the skill or run runtime probes.
If the agent misinterprets a task or is given unsafe instructions, it could click buttons, type text, submit forms, close windows, or trigger changes in visible apps and accounts on the user’s Mac.
These are broad desktop-control primitives. The artifacts disclose them, but do not bound which apps/actions may be controlled or require confirmation before high-impact clicks, typing, closing windows, launching apps, or submitting changes.
- **Mouse Control**: Click, double-click, right-click, move, drag - **Keyboard Input**: Type text, press keys, shortcuts - **Window Management**: List windows, focus, resize, close - **App Control**: Launch, quit, bring to front
Use only in supervised sessions. Add or follow a policy requiring explicit confirmation before destructive actions, form submissions, payments, account changes, credential entry, or actions outside a user-named app/task.
Granting these permissions can let the agent see sensitive on-screen information and interact with many apps as the logged-in user.
The skill asks the user to grant Accessibility and Screen Recording permissions to the terminal/OpenClaw runtime. That is expected for GUI automation, but it gives the agent broad ability to observe the screen and control the UI.
在 **系统设置 → 隐私与安全性 → 辅助功能** 中添加： - Terminal / iTerm - 你用来运行 OpenClaw 的终端应用 ### 2. 屏幕录制权限（截图需要）
Grant these macOS permissions only if you trust the skill and need GUI automation. Consider revoking Accessibility and Screen Recording access when finished.
Sensitive screen contents may remain in /tmp/gui-auto/screen.png or enter the agent conversation/context after OCR.
The helper writes screen captures to a reusable temporary directory and emits OCR text to the agent. This is purpose-aligned, but screenshots and OCR may contain passwords, private messages, or other sensitive visible data, and no cleanup guidance is provided.
SCREENSHOT_DIR="/tmp/gui-auto" mkdir -p "$SCREENSHOT_DIR" ... screencapture "$SCREENSHOT_DIR/screen.png" ... tesseract "$img" stdout
Avoid using the skill while secrets are visible, and delete /tmp/gui-auto/screen.png after sensitive sessions. The skill author should add cleanup and retention guidance.
The skill may fail unless the right macOS tools are installed, and users may need to install dependencies from sources outside this reviewed artifact set.
The skill relies on local GUI/OCR tools, while the registry metadata declares no required binaries, no install spec, and no OS restriction. This is not malicious by itself, but dependency and platform requirements are under-documented.
### cliclick (Mouse/Keyboard) ... ### screencapture + tesseract (Screen Reading) ... ### osascript (AppleScript - Window/App Control)
Document and declare required binaries and macOS-only support, and install dependencies from trusted, pinned sources.