Openclaw Mlx Audio

Local TTS/STT integration for OpenClaw using mlx-audio - Zero API keys, Zero cloud dependency

Security Scan

The skill broadly matches a local TTS/STT plugin, but there are mismatches between claimed architecture and shipped files plus an external install path (curl | sh / uv tool), so proceed with caution and review before running installers.

Purpose & Capability

The stated purpose (local TTS/STT via mlx-audio) matches the plugin code: the JS plugin invokes local mlx_audio CLI commands and there are Python helper servers and tests for local TTS/STT. Minor mismatch: SKILL metadata and README emphasize a CLI-only approach, but the package also includes python-runtime/tts_server.py and stt_server.py and openclaw.plugin.json contains ports/pythonEnvMode settings — this is inconsistent (CLI-only claim vs shipped Python servers).

Instruction Scope

SKILL.md instructs local installation (brew, uv, ffmpeg, uv tool install mlx-audio) and copying the extension into ~/.openclaw/extensions. The runtime code indeed executes local CLI commands and reads/writes temp audio files. Concern: the plugin's runtime checkDependencies requires mlx_audio.tts.generate and mlx_audio.stt.generate even though those CLI binaries are not listed in the SKILL.md 'required bins' metadata (only brew, ffmpeg, uv are listed). Also SKILL.md and some docs repeatedly say "No Python API" while python runtime servers are present — that discrepancy grants the code broader runtime surface than the prose describes. The Python servers accept HTTP uploads (local-only but configurable host/port) and parse multipart bodies, which expands the attack/abuse surface relative to a pure CLI plugin.

Install Mechanism

There is no registry install spec, but an included install.sh performs network installs: it runs curl -LsSf https://astral.sh/uv/install.sh (pipelines a remote installer) and then uses 'uv tool install --prerelease=allow' to fetch mlx-audio from the uv ecosystem. Using curl|sh and tool-installers pulls code at install time from external servers — common for CLI managers but higher risk than pure package-manager installs. The script also offers apt/dnf/brew branches (multi-platform) despite the repo declaring darwin-only. The install step will create binaries and run them locally (extract/install).

Credentials

The skill declares no required env vars or credentials (good). The code does read or forward process.env into spawned processes and the Python servers accept STT_MODEL / STT_LANGUAGE env vars (optional overrides). There are no requests for API keys or unrelated cloud credentials. Passing the agent's environment to executed CLI processes is expected but worth noting because any secrets in the agent env would be visible to child processes.

Persistence & Privilege

The registry flags are normal (always: false) and the skill does not request elevated system permissions in metadata. It installs into the user's OpenClaw extension directory per SKILL.md/install.sh which is expected for a plugin. It does not attempt to modify other skills or global agent configuration in the provided files.

What to consider before installing

What to consider before installing: - Verify the upstream source: SKILL.md and docs reference a GitHub repo; confirm the repository (and commit history) before running install.sh. The registry record lacks a homepage, so validate the source yourself. - Inspect install.sh and the external installer URLs (astral.sh and uv tool) before running. The installer uses curl | sh and uv tool install to fetch mlx-audio binaries — this will download and execute code from the network. If you are uncomfortable, download the repo and examine artifacts offline or run install in a sandbox/VM. - Decide whether you need the Python servers. The package claims CLI-only in some places but includes python-runtime/*.py (HTTP endpoints that accept uploads). If you want minimal attack surface, disable or avoid starting the Python servers and use the CLI-only paths. - Check what environment variables your agent exposes. The plugin spawns child processes that inherit process.env; any sensitive env values accessible to the agent could be seen by the invoked CLI tools. Remove or isolate secrets where possible before running. - Run the tests locally in a controlled environment (bash test/run_tests.sh) to verify behavior. Prefer manual verification of mlx_audio CLI commands (which the plugin will call) before integrating into your main OpenClaw instance. - If you require higher assurance, ask the maintainer for an authoritative repo URL and a signed release or for an explanation of why Python servers are present despite CLI-only claims. That will raise confidence and resolve the architectural inconsistency. Additional info that would raise confidence: a public GitHub repo with release tags, commit history, and community reviews; an install path that avoids curl|sh (or a documented, auditable release tarball); clarification whether python-runtime servers are optional and how to disable them.

本地支持多语言文本转语音 (TTS) 和语音转文本 (STT)，完全在 Apple Silicon 设备上运行，无需云服务，保护数据隐私。

功能

• 🗣️ TTS 文本转语音: 支持中文、英文等多种语言
• 🎤 STT 语音转文本: 高准确率语音识别
• 🎭 声音克隆: 使用参考音频克隆声音
• 🔒 完全本地: 无需 API Key，数据不出设备

安装

# 安装依赖
brew install ffmpeg uv
uv tool install mlx-audio --prerelease=allow

# 安装插件
cp -r openclaw-mlx-audio ~/.openclaw/extensions/

# 重启 OpenClaw
openclaw gateway restart

使用

TTS 命令

# 状态查询
/ mlx-tts status

# 测试生成
/ mlx-tts test "你好，这是测试语音"

# 模型列表
/ mlx-tts models

STT 命令

# 状态查询
/ mlx-stt status

# 转录音频
/ mlx-stt transcribe /path/to/audio.wav

# 模型列表
/ mlx-stt models

工具调用

TTS:

{
  "tool": "mlx_tts",
  "parameters": {
    "action": "generate",
    "text": "Hello World",
    "outputPath": "/tmp/speech.mp3"
  }
}

STT:

{
  "tool": "mlx_stt",
  "parameters": {
    "action": "transcribe",
    "audioPath": "/tmp/audio.wav",
    "language": "zh"
  }
}

支持模型

TTS 模型

模型	语言	速度	质量
mlx-community/Kokoro-82M-bf16	8+	⚡⚡⚡	Good
mlx-community/Qwen3-TTS-12Hz-0.6B-Base-bf16	ZH/EN/JA/KO	⚡⚡	Better
mlx-community/Qwen3-TTS-12Hz-1.7B-VoiceDesign-bf16	ZH/EN/JA/KO	⚡	Best

STT 模型

模型	语言	速度	精度
mlx-community/whisper-large-v3-turbo-asr-fp16	99+	⚡⚡⚡	Good
mlx-community/Qwen3-ASR-1.7B-8bit	ZH/EN/JA/KO	⚡⚡	Better
mlx-community/whisper-large-v3	99+	⚡⚡	Best

测试

自动化测试: 17 项 (100% 通过) 真人测试: 11 项 Discord 测试 总体评分: ⭐⭐⭐⭐ (3.85/5.0)

运行测试:

bash test/run_tests.sh

配置

在 openclaw.json 中添加:

{
  "plugins": {
    "allow": ["@openclaw/mlx-audio"],
    "entries": {
      "@openclaw/mlx-audio": {
        "enabled": true,
        "config": {
          "tts": {
            "enabled": true,
            "model": "mlx-community/Qwen3-TTS-12Hz-0.6B-Base-bf16",
            "langCode": "zh"
          },
          "stt": {
            "enabled": true,
            "model": "mlx-community/Qwen3-ASR-1.7B-8bit",
            "language": "zh"
          }
        }
      }
    }
  }
}

系统要求

• macOS Apple Silicon (M1/M2/M3)
• Node.js 18+
• Python 3.10+
• ffmpeg
• uv

链接

• GitHub: https://github.com/gandli/openclaw-mlx-audio
• OpenClaw: https://docs.openclaw.ai
• ClawHub: https://clawhub.ai

License

MIT