{"skill":{"slug":"pyautogui-controller","displayName":"PyAutoGUI Controller","summary":"Control Windows desktop with natural-language commands for screenshots, app launching, typing, clicking, and multi-step UI automation using a local Python co...","description":"---\nname: pyautogui-controller\ndescription: Advanced Windows desktop/browser automation via a local Python controller project. Use when the user wants natural-language UI automation on this machine: screenshots, opening apps, browser navigation, clicking inputs/buttons, typing text, and simple multi-step desktop workflows. Prefer built-in browser/web tools for normal web tasks; use this skill when direct screen/desktop control is required.\nmetadata: { \"openclaw\": { \"emoji\": \"🤖\", \"os\": [\"win32\"], \"requires\": { \"anyBins\": [\"python\", \"python3\"] } } }\n---\n\n# PyAutoGUI Controller\n\nUse this skill when the user needs **local desktop automation on this Windows machine** and normal browser tools are not enough.\n\n## Use for\n\n- Open desktop apps and interact with their windows\n- Take screenshots\n- Type Chinese/English text into native apps\n- Click buttons/inputs on screen\n- Run short natural-language automation commands through the local controller\n- Browser actions that require screen-based control instead of DOM/web APIs\n\n## Avoid for\n\n- Normal web reading/searching → use `web_fetch`, `browser`, or search skills\n- Mobile automation\n- High-assurance unattended long workflows\n\n## Project location\n\nThe controller project lives at:\n\n`C:\\Users\\dev\\Desktop\\昱昱\\skills\\pyautogui-controller`\n\n## Primary invocation\n\nRun the local controller through the wrapper script:\n\n```powershell\npython {baseDir}\\scripts\\run_controller.py \"打开记事本，然后输入 '你好'\"\n```\n\nThe wrapper calls the real project entrypoint:\n\n`C:\\Users\\dev\\Desktop\\昱昱\\skills\\pyautogui-controller\\main.py`\n\n## Recommended workflow\n\n1. Prefer a **single compound command** over multiple split commands\n2. Start with simple actions\n3. Inspect returned JSON for `success`, `results`, `runtime`, and `warnings`\n4. If the command fails, simplify the wording and retry\n\n## Good command examples\n\n```powershell\npython {baseDir}\\scripts\\run_controller.py \"截图\"\npython {baseDir}\\scripts\\run_controller.py \"打开记事本\"\npython {baseDir}\\scripts\\run_controller.py \"打开浏览器访问 https://example.com\"\npython {baseDir}\\scripts\\run_controller.py \"打开浏览器访问 file:///C:/path/test.html，然后在搜索框输入 'abc'，再点击发送按钮\"\n```\n\n## Practical notes\n\n- Compound commands are more reliable than split follow-ups\n- OCR / DOM readiness may affect accuracy; check `warnings`\n- Move the mouse to the top-left corner if PyAutoGUI fail-safe is triggered\n- This skill is local-machine specific; it assumes the project path above exists\n","topics":["Browser Automation"],"tags":{"latest":"1.0.1"},"stats":{"comments":0,"downloads":1044,"installsAllTime":39,"installsCurrent":2,"stars":0,"versions":2},"createdAt":1773536178772,"updatedAt":1778999827916},"latestVersion":{"version":"1.0.1","createdAt":1773536210829,"changelog":"Public upload from desktop skill folder","license":"MIT-0"},"metadata":null,"owner":{"handle":"hzj520lyx","userId":"s17dmedxkp23t4epx29gbkxpvn8417a0","displayName":"hzj520lyx","image":"https://avatars.githubusercontent.com/u/78837655?v=4"},"moderation":{"isSuspicious":false,"isMalwareBlocked":false,"verdict":"clean","reasonCodes":["review.llm_review"],"summary":"Review: review.llm_review","engineVersion":"v2.4.24","updatedAt":1780089890843}}