ClawHub

Audio Intelligence

Other

Configure and use Gladia audio intelligence features: speaker diarization, translation, sentiment analysis, named entity recognition (NER), PII redaction, subtitles (SRT/VTT), summarization, chapterization, custom vocabulary, and audio-to-LLM. Use when the user asks about any audio intelligence feature, enabling features on pre-recorded or live transcription, understanding which features are available in each mode, or combining multiple features. Always prefer the official SDK; fall back to raw REST only when SDK cannot satisfy the requirement.

Install

openclaw skills install audio-intelligence

Audio Intelligence

Gladia's audio intelligence features extract structured data and insights from transcripts. They work on top of the base transcription — most are enabled by adding options to the transcribe() call (pre-recorded) or the startSession() config (live).

SDK-first: always use the official SDK — see sdk-integration for policy, setup, and fallback criteria.

When to Use

  • User asks about a specific feature: diarization, translation, PII redaction, sentiment, NER, subtitles, summarization, etc.
  • Enabling or configuring one or more audio intelligence features on pre-recorded or live transcription
  • Understanding which features are available in live vs pre-recorded mode
  • Combining multiple features in a single transcription job

When NOT to use: For basic transcription without audio intelligence features, go directly to pre-recorded-transcription or live-transcription. For gotchas and errors related to specific features, see troubleshooting.

References

Consult these resources as needed:

  • ./references/live-audio-intelligence.md -- Detailed config and WebSocket responses for all live-mode features
  • ./references/pre-recorded-audio-intelligence.md -- Detailed config and response structures for all pre-recorded audio intelligence features
  • ../pre-recorded-transcription/SKILL.md -- Pre-recorded transcription workflow and options
  • ../live-transcription/SKILL.md -- Live transcription session config and event handling
  • ../sdk-integration/SKILL.md -- SDK setup, client initialization, and SDK vs raw API decision guide
  • ../troubleshooting/SKILL.md -- Common errors, gotchas, and verification checklist

Feature Availability

FeaturePre-recordedLiveConfig key
Speaker diarizationYesNodiarization
TranslationYesYestranslation
Sentiment analysisYesYessentiment_analysis
Named entity recognitionYesYesnamed_entity_recognition
Subtitles (SRT/VTT)YesNosubtitles
Custom vocabularyYesYescustom_vocabulary
PII redactionYesNopii_redaction
ChapterizationYesYeschapterization (post-process)
SummarizationYesYessummarization (post-process)
Audio-to-LLMYesNoaudio_to_llm
Custom spellingYesYescustom_spelling
Custom metadataYesYescustom_metadata

Live features split into two groups: real-time (results stream during the session) and post-processing (results arrive after stopRecording()). See ./references/live-audio-intelligence.md for details.

Quick Config Examples

Code examples assume GladiaClient is already initialized — see sdk-integration for setup.

Speaker Diarization (pre-recorded only)

const result = await client.preRecorded().transcribe("audio.mp3", {
  diarization: true,
  diarization_config: { number_of_speakers: 2 },
});
// Each utterance includes a `speaker` field (0-indexed integer)

result = client.prerecorded().transcribe("audio.mp3", {
    "diarization": True,
    "diarization_config": {"number_of_speakers": 2},
})

Translation (pre-recorded and live)

Pre-recorded:

const result = await client.preRecorded().transcribe("audio.mp3", {
  translation: true,
  translation_config: { target_languages: ["fr", "es"] },
});

result = client.prerecorded().transcribe("audio.mp3", {
    "translation": True,
    "translation_config": {"target_languages": ["fr", "es"]},
})

Live (result streams as translation WebSocket events — see live-audio-intelligence.md):

const session = client.liveV2().startSession({
  // ... audio format options ...
  realtime_processing: {
    translation: true,
    translation_config: { target_languages: ["fr"] },
  },
});

from gladiaio_sdk import LiveV2InitRequest, LiveV2RealtimeProcessing

session = client.live().start_session(
    LiveV2InitRequest(
        # ... audio format options ...
        realtime_processing=LiveV2RealtimeProcessing(
            translation=True,
            translation_config={"target_languages": ["fr"]},
        ),
    )
)

Summarization (pre-recorded and live)

Pre-recorded:

const result = await client.preRecorded().transcribe("audio.mp3", {
  summarization: true,
  summarization_config: { type: "bullet_points" },
});

Live (arrives after stopRecording() as post_summarization event):

const session = client.liveV2().startSession({
  // ... audio format options ...
  post_processing: {
    summarization: true,
    summarization_config: { type: "bullet_points" },
  },
});
session.on("message", (msg) => {
  if (msg.type === "post_summarization") console.log(msg.data.results);
});

For full per-feature config options and response structures, see:

Common Mistakes

  • code_switching: true with empty languages: triggers evaluation across 100+ languages and causes frequent misdetections. Always provide 3-5 expected languages.
  • Custom vocabulary intensity above 0.6: values over 0.6 cause false positives where unrelated words get replaced. Keep at 0.4-0.6 and use pronunciations for better results.
  • Expecting diarization, PII redaction, subtitles, or audio-to-LLM in live mode: these four features are pre-recorded only.
  • Enabling many features simultaneously without considering cost/latency: each enabled feature adds processing time. Enable only what you need; combine diarization + summarization + translation only when all are required.

For the full gotcha list, see troubleshooting.

Further Reading