TUI USE

Operate TUI (terminal user interface) programs through the tui-agent CLI tool. Use this skill whenever the user asks to interact with, automate, control, or...

Security Scan

The skill's purpose (automating terminal UIs via a tui-agent CLI) is coherent, but the SKILL.md expects you to install and run an external tui-agent implementation from a GitHub path without the registry declaring that dependency — this mismatch and the informal install guidance are worth caution.

Purpose & Capability

The name/description match the instructions: the skill is an instruction set for controlling TUI programs via a CLI called tui-agent. However, the registry metadata declares no required binaries while SKILL.md explicitly requires a 'tui-agent' CLI (and points to a GitHub repo). The missing declared dependency is an inconsistency: a legitimate TUI automation skill should list the CLI as a required binary or provide a proper install spec.

Instruction Scope

The SKILL.md stays on-topic: it documents starting sessions, capturing screens, sending keys, waiting for text, etc., and does not instruct reading arbitrary system files or exporting credentials. That said, the skill presumes the agent will run arbitrary terminal programs (editors, database clients, monitors) — these programs can expose sensitive data on screen. The instructions also point users to obtain code from a third-party GitHub repo and to run pip install -e tui-agent/, which is an action that downloads and executes external code (outside the skill bundle).

Install Mechanism

There is no install spec in the registry (instruction-only), but SKILL.md tells the user to clone or use a GitHub tree URL and run 'pip install -e tui-agent/'. That guidance is ambiguous (pip install -e expects a local path) and directs installing code from an external GitHub repository (zbflcy/tui-use). Installing arbitrary code from a third-party repo is higher risk and should be declared in install metadata or replaced with a clear, verifiable release URL or package. Because the skill encourages installing/executing external code without provenance, this is a concern.

Credentials

The skill declares no required environment variables, no credentials, and no config paths. The SKILL.md likewise does not request secrets. This is proportionate. Note: automating terminal programs inherently gives the operator (or an agent using this skill) visibility into anything printed to the terminal (which could include secrets), so operational caution is warranted even though no explicit secret access is requested.

Persistence & Privilege

Flags show no 'always: true' and the skill is user-invocable only. Autonomous model invocation is allowed by default but not an additional privilege requested here. The skill does refer to a daemon started by the tui-agent runtime, but that daemon is part of the external tool rather than the skill package itself. No evidence the skill tries to modify other skills or system-wide agent settings.

What to consider before installing

Before installing or using this skill: 1) verify you actually have a trusted 'tui-agent' binary available (the registry metadata should have declared this dependency but it doesn't). 2) Do not blindly run the SKILL.md's install advice; the repo URL points to an arbitrary GitHub user and the suggested 'pip install -e tui-agent/' is ambiguous — prefer an official release package or inspect the repository contents before installing. 3) Understand that automating terminal programs lets the tool see everything shown in the terminal (potentially sensitive), and the external tui-agent daemon will spawn PTYs and run commands — audit the code if you will install it. 4) If you proceed, restrict usage to manual invocation (do not enable any always-on/autonomous behaviors for untrusted skills), and consider running the tool in a sandboxed environment until you confirm its safety.

TUI-Use: Operating TUI Programs via tui-agent

This skill teaches you how to use the tui-agent CLI tool to start, interact with, and control TUI (Terminal User Interface) programs. Think of tui-agent as your eyes and hands for terminal programs — it lets you see what's on screen, type text, press keys, click with the mouse, and wait for things to happen.

Prerequisites

tui-agent must be installed and available in PATH, you can clone from tui-use github. Install from the sibling tui-agent/ directory:

pip install -e tui-agent/

Architecture at a Glance

You (via Bash tool)  -->  tui-agent CLI  -->  Daemon (auto-started)  -->  PTY  -->  TUI Program
                     <--  stdout/JSON    <--  Unix Socket             <--  pyte virtual screen

The daemon starts automatically on first use — no manual setup needed.

Core Workflow

Every TUI interaction follows this pattern:

1. Start a session (launches the TUI program)
2. Wait for the program to be ready
3. Capture the screen to see what's displayed
4. Interact (type text, send keys, click, scroll)
5. Capture again to verify the result
6. Stop the session when done

This capture-act-capture loop is the fundamental rhythm. Always capture the screen before and after important actions so you can verify what happened.

Command Reference

Session Management

Start a TUI program:

tui-agent start --name <session-name> [--cols 120] [--rows 40] -- <command> [args...]

• --name: A unique identifier for this session (you'll use it in all subsequent commands)
• --cols/--rows: Terminal dimensions (default: 80x24). Use larger values for complex UIs
• Everything after -- is the command to run

List active sessions:

tui-agent list

Check session status:

tui-agent status --name <session-name>

Stop a session:

tui-agent stop --name <session-name>
# Or stop everything:
tui-agent stop --all

Seeing the Screen

Capture full screen:

tui-agent capture --name <session-name>

Capture with cursor position:

tui-agent capture --name <session-name> --cursor

This appends cursor coordinates to the output — helpful for knowing where you are in an editor.

Capture a specific region:

tui-agent capture --name <session-name> --top 0 --left 0 --height 5 --width 40

Save a snapshot for later comparison:

tui-agent capture --name <session-name> --save my_snapshot

Compare current screen to a saved snapshot:

tui-agent diff --name <session-name> --snapshot my_snapshot

Read scrollback history (content that scrolled off the top):

tui-agent scrollback --name <session-name> --lines 200

Typing and Keys

Type text:

tui-agent type --name <session-name> "hello world"

Type text and press Enter:

tui-agent type --name <session-name> --enter "ls -la"

Send special keys:

tui-agent key --name <session-name> <key1> [key2] [key3...]

Important: key vs type — The key command only accepts special key names listed below. For regular characters (letters, numbers, symbols) like vim's i, o, dd, :wq, always use type. Use key only for keys that aren't printable characters.

Supported key names (case-insensitive):

• Navigation: up, down, left, right, home, end, pageup, pagedown
• Editing: enter/return, tab, backspace, delete, insert, space, escape/esc
• Function keys: f1 through f12
• Ctrl combos: ctrl-a through ctrl-z

Paste text (uses bracketed paste mode, safe for editors):

tui-agent paste --name <session-name> "multi-line\ntext content"

Use paste instead of type when inserting multi-line content into editors — bracketed paste mode prevents the editor from interpreting line breaks as commands.

Mouse Operations

Click at a position (0-indexed row and column):

tui-agent click --name <session-name> --row 5 --col 10

Scroll up/down:

tui-agent scroll-up --name <session-name> --row 12 --col 40 --lines 3
tui-agent scroll-down --name <session-name> --row 12 --col 40 --lines 3

Waiting for Conditions

Waiting is critical for reliable automation. TUI programs take time to render, and acting too fast leads to errors.

Wait for specific text to appear on screen:

tui-agent wait --name <session-name> --text "Ready" --timeout 10

Wait for text to disappear:

tui-agent wait --name <session-name> --text "Loading..." --absent --timeout 15

Wait for screen to stop changing (stable for N seconds):

tui-agent wait --name <session-name> --stable 2.0 --timeout 10

Use --stable when you don't know exactly what text to expect, but need the program to finish rendering.

Resizing the Terminal

tui-agent resize --name <session-name> --cols 160 --rows 50

Best Practices

Always wait before capturing

TUI programs need time to render. After starting a session or sending input, always wait before capturing:

tui-agent start --name ed -- vim file.py
tui-agent wait --name ed --stable 1.0         # Wait for vim to fully load
tui-agent capture --name ed --cursor           # Now safe to read the screen

Use meaningful session names

Name sessions after their purpose, not the program:

• Good: editor, monitor, file-browser
• Bad: s1, test, abc

Capture before and after actions

This is how you verify that your actions had the intended effect:

tui-agent capture --name ed --save before_edit
tui-agent type --name ed "i"                   # Enter insert mode (single char, use type)
tui-agent type --name ed "new line of code"
tui-agent key --name ed escape                 # Escape is a special key, use key
tui-agent capture --name ed                    # Check the result

Choose appropriate terminal size

• For simple programs (bash, basic TUIs): 80x24 (default) is fine
• For editors or complex UIs: use 120x40 or larger
• For programs with wide tables: increase cols to 160+

Clean up sessions when done

Always stop sessions you no longer need:

tui-agent stop --name ed

Or stop all sessions if you're done:

tui-agent stop --all

Handle errors gracefully

If a wait command times out (exit code 1), capture the screen to understand what happened instead of blindly retrying:

tui-agent wait --name ed --text "Success" --timeout 5
# If this fails, capture to see the actual state:
tui-agent capture --name ed

Common Workflows

Editing a File with nano

# Start nano
tui-agent start --name ed --cols 120 --rows 40 -- nano myfile.py
tui-agent wait --name ed --stable 1.0
tui-agent capture --name ed

# Type some code
tui-agent type --name ed "print('hello world')"
tui-agent key --name ed enter
tui-agent type --name ed "print('goodbye')"
tui-agent capture --name ed

# Save (Ctrl-O, then Enter to confirm filename)
tui-agent key --name ed ctrl-o
tui-agent wait --name ed --stable 0.5
tui-agent key --name ed enter
tui-agent wait --name ed --text "Wrote"

# Exit (Ctrl-X)
tui-agent key --name ed ctrl-x
tui-agent stop --name ed

Editing a File with vim

Note: vim may not respond to input on some platforms (see Platform Notes). If vim works in your environment:

# Start vim (use -u NONE to avoid config issues)
tui-agent start --name ed --cols 120 --rows 40 -- vim -u NONE -N myfile.py
tui-agent wait --name ed --stable 1.0
tui-agent capture --name ed

# Navigate to a specific line (e.g., line 10)
tui-agent type --name ed ":10"
tui-agent key --name ed enter
tui-agent capture --name ed

# Enter insert mode and add text
tui-agent type --name ed "o"                    # Open new line below (normal char, use type)
tui-agent type --name ed "    print('hello')"
tui-agent key --name ed escape                  # Back to normal mode

# Save and quit
tui-agent type --name ed ":wq"
tui-agent key --name ed enter
tui-agent wait --name ed --stable 1.0

Monitoring System with htop

# Start htop
tui-agent start --name mon --cols 160 --rows 50 -- htop
tui-agent wait --name mon --stable 2.0
tui-agent capture --name mon

# Search for a process
tui-agent key --name mon f3                     # F3 is a special key, use key
tui-agent type --name mon "python"
tui-agent key --name mon enter
tui-agent capture --name mon

# Scroll through the list
tui-agent scroll-down --name mon --row 25 --col 80 --lines 10
tui-agent capture --name mon

# Quit
tui-agent type --name mon "q"                   # Regular char, use type
tui-agent stop --name mon

Running a Shell Command and Reading Output

# Start a bash session
tui-agent start --name sh -- bash --norc --noprofile
tui-agent wait --name sh --stable 1.0

# Run a command
tui-agent type --name sh --enter "ls -la /tmp"
tui-agent wait --name sh --stable 1.0
tui-agent capture --name sh

# If output is long, check scrollback
tui-agent scrollback --name sh --lines 100

# Exit
tui-agent type --name sh --enter "exit"
tui-agent stop --name sh

Interacting with a ncurses Application

# Start any ncurses app (example: midnight commander)
tui-agent start --name fm --cols 120 --rows 40 -- mc
tui-agent wait --name fm --stable 2.0
tui-agent capture --name fm

# Navigate using arrow keys
tui-agent key --name fm down down down
tui-agent capture --name fm

# Press enter to open a directory
tui-agent key --name fm enter
tui-agent wait --name fm --stable 1.0
tui-agent capture --name fm

# Use function keys
tui-agent key --name fm f5                      # Copy dialog
tui-agent capture --name fm

# Quit
tui-agent key --name fm f10
tui-agent stop --name fm

Platform Notes

vim on macOS: The system vim (/usr/bin/vim) may not respond to type input when launched directly via tui-agent start. This appears to be related to how macOS vim detects terminal capabilities. Workarounds:

• Use nano or pico instead — they work reliably with tui-agent
• Use vim -u NONE -N to disable vimrc and run in nocompatible mode
• Install a different vim build (e.g., via Homebrew: brew install vim)
• For simple file edits, consider using shell commands (sed, awk) instead of a TUI editor

Other programs: Most terminal programs (bash, htop, mc, ncurses apps) work well out of the box with tui-agent. If a program doesn't respond to input, try increasing the wait time or using --stable with a longer duration.

Troubleshooting

"session not found" error: The session may have been stopped or the program exited. Check with tui-agent list.

Screen capture is empty or garbled: The program may not have finished rendering. Add a longer --stable wait or increase --timeout.

Keys don't seem to work: Some programs expect specific key sequences. Capture the screen to verify the program's current state (e.g., is vim in insert mode or normal mode?).

Program exited unexpectedly: Use tui-agent status --name <session> to check the exit code. The program may have crashed or received a signal.

Daemon not running: The daemon starts automatically. If issues persist, check if the socket file exists at /tmp/tui-agent-*.sock. You can manually start the daemon or simply retry the command.