Mac Compute Use

Control macOS applications via Accessibility API through an MCP server. Open apps, click buttons, type text, press keys, scroll, and read UI state. Use when: (1) interacting with native macOS apps (Finder, Messages, Mail, etc.), (2) automating GUI workflows — clicking, typing, navigating menus, (3) reading screen content or UI state from any app, (4) controlling browsers for web automation without browser extensions, (5) any task requiring 'computer use' or 'desktop control' on macOS. NOT for: Linux/Windows, headless servers, or tasks achievable via CLI/API without GUI.

Control macOS GUI through the Accessibility API via MCP.

Setup

1. Install the MCP server:

brew tap reedburns/mcp-server-macos-use
brew install mcp-server-macos-use

2. Register with mcporter:

mcporter config add macos-use --transport stdio --command $(which mcp-server-macos-use)

3. Grant Accessibility permission: System Settings → Privacy & Security → Accessibility → add mcp-server-macos-use

4. Verify:

mcporter list macos-use --schema

Tools

All tools are called via mcporter call macos-use.<tool> key=value.

open_application_and_traverse

Open/activate an app and get its UI tree.

mcporter call macos-use.macos-use_open_application_and_traverse identifier="Google Chrome"
mcporter call macos-use.macos-use_open_application_and_traverse identifier="com.apple.finder"
mcporter call macos-use.macos-use_open_application_and_traverse identifier="TextEdit"

Returns: PID, element count, visible interactive elements, and a JSON file path with full UI tree.

click_and_traverse

Click at coordinates (from UI tree) and get updated state.

mcporter call macos-use.macos-use_click_and_traverse pid=408 x=701 y=73 width=102 height=41

• x, y: top-left corner of the element (from traversal)
• width, height: optional, when provided click lands at center

type_and_traverse

Type text into the focused app.

mcporter call macos-use.macos-use_type_and_traverse pid=408 text="Hello world"

press_key_and_traverse

Press a key with optional modifiers.

mcporter call macos-use.macos-use_press_key_and_traverse pid=408 keyName=Return
mcporter call macos-use.macos-use_press_key_and_traverse pid=408 keyName=a modifierFlags='["Command"]'
mcporter call macos-use.macos-use_press_key_and_traverse pid=408 keyName=Tab
mcporter call macos-use.macos-use_press_key_and_traverse pid=408 keyName=Escape

Valid modifiers: CapsLock, Shift, Control, Option, Command, Function, NumericPad, Help.

scroll_and_traverse

Scroll within an app window.

mcporter call macos-use.macos-use_scroll_and_traverse pid=408 x=500 y=400 deltaY=3
mcporter call macos-use.macos-use_scroll_and_traverse pid=408 x=500 y=400 deltaY=-3

• deltaY positive = scroll down, negative = scroll up
• deltaX optional, for horizontal scroll

refresh_traversal

Get current UI state without performing any action.

mcporter call macos-use.macos-use_refresh_traversal pid=408

Workflow Pattern

Typical automation flow:

1. Open app → get PID and visible elements
2. Read the visible_elements in the response summary — these are interactive elements with coordinates
3. Click/type/press using coordinates from the UI tree
4. Read the response — it shows what changed (diff) and new visible elements
5. Repeat until task is complete

Reading the Response

Each tool returns a compact summary with:

• status: success/error
• pid: process ID (use for subsequent calls)
• file: path to full JSON with all elements (use grep or python3 to search)
• visible_elements: key interactive elements currently visible, with coordinates

When you need to find a specific element, grep the JSON file:

grep -i "search text" /tmp/macos-use/<file>.json

Or parse with Python:

python3 -c "
import json
with open('/tmp/macos-use/<file>.json') as f:
    data = json.load(f)
for e in data.get('traversal',{}).get('elements',[]):
    text = (e.get('text') or '').strip()
    if text and 'search' in text.lower():
        print(f'[{e[\"role\"]}] ({e[\"x\"]},{e[\"y\"]} {e.get(\"width\",\"?\")}x{e.get(\"height\",\"?\")}) {text}')
"

Tips

• Always use --output json for machine-readable results when chaining commands
• After clicking, wait a moment then refresh_traversal if the UI didn't update in the diff
• Use app name ("Google Chrome"), bundle ID ("com.google.Chrome"), or path to open apps
• Coordinates are absolute screen positions — if the window moves, refresh the traversal
• The server writes traversal JSON to /tmp/macos-use/ — these files are temporary