AWS Transcribe Streaming in MCP Servers — Real-Time Audio Transcription, EventStream, Partial Results

Package and client setup

// npm install @aws-sdk/client-transcribe-streaming
// This is a DIFFERENT package from @aws-sdk/client-transcribe (batch)

import {
  TranscribeStreamingClient,
  StartStreamingTranscriptionCommand,
  TranscriptResultStream,
} from '@aws-sdk/client-transcribe-streaming';

// The streaming client uses HTTP/2 — it needs a region with streaming support
// Streaming is available in: us-east-1, us-east-2, us-west-2, eu-west-1,
// eu-central-1, ap-southeast-2, ap-northeast-1, ca-central-1
const streamingClient = new TranscribeStreamingClient({ region: 'us-east-1' });

The streaming SDK uses HTTP/2 bidirectional streams under the hood (not WebSockets). The Node.js SDK handles the transport — you interact with it through async generators for input and async iterators for output. Do not share a single TranscribeStreamingClient across concurrent streaming sessions; create a new client per session or use a pool.

Basic streaming tool

import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
import { z } from 'zod';
import { readFile } from 'fs/promises';

const server = new McpServer({ name: 'streaming-transcribe', version: '1.0.0' });

server.tool('transcribe_audio_stream', {
  // In a real MCP server the audio would come from an S3 key or a tool-provided buffer.
  // This example reads a local PCM file for illustration.
  audio_file_path: z.string().describe('Path to raw 16kHz 16-bit PCM audio file'),
  language_code: z.string().default('en-US'),
  show_speakers: z.boolean().default(false),
}, async ({ audio_file_path, language_code, show_speakers }) => {
  // Load audio into memory (replace with streaming source for large files)
  const audioBuffer = await readFile(audio_file_path);

  // Chunk the audio into 100 ms frames at 16kHz 16-bit mono = 3200 bytes/frame
  const FRAME_BYTES = 3200;
  async function* audioStream() {
    for (let offset = 0; offset < audioBuffer.length; offset += FRAME_BYTES) {
      yield {
        AudioEvent: {
          AudioChunk: audioBuffer.subarray(offset, offset + FRAME_BYTES),
        },
      };
      // Add a small delay so we don't overwhelm the stream with all chunks at once
      await new Promise(r => setTimeout(r, 1));
    }
  }

  const command = new StartStreamingTranscriptionCommand({
    LanguageCode: language_code,
    MediaEncoding: 'pcm',
    MediaSampleRateHertz: 16000,
    AudioStream: audioStream(),
    EnablePartialResultsStabilization: true,
    PartialResultsStability: 'high',
    ShowSpeakerLabel: show_speakers,
  });

  const { TranscriptResultStream: resultStream } =
    await streamingClient.send(command);

  const finalSegments: string[] = [];
  const speakerTurns: { speaker: string; text: string }[] = [];

  for await (const event of resultStream as AsyncIterable<any>) {
    if (!event.TranscriptEvent) continue;

    for (const result of event.TranscriptEvent.Transcript.Results ?? []) {
      if (result.IsPartial) continue; // Skip partial results — wait for the final

      const transcript = result.Alternatives?.[0]?.Transcript ?? '';
      finalSegments.push(transcript);

      if (show_speakers) {
        const items = result.Alternatives?.[0]?.Items ?? [];
        let currentSpeaker = '';
        let words: string[] = [];
        for (const item of items) {
          if (item.Type !== 'pronunciation') continue;
          const spk = item.Speaker ?? 'spk_0';
          if (spk !== currentSpeaker) {
            if (words.length) speakerTurns.push({ speaker: currentSpeaker, text: words.join(' ') });
            currentSpeaker = spk;
            words = [];
          }
          words.push(item.Content);
        }
        if (words.length) speakerTurns.push({ speaker: currentSpeaker, text: words.join(' ') });
      }
    }
  }

  return {
    content: [{
      type: 'text',
      text: JSON.stringify({
        transcript: finalSegments.join(' '),
        speakers: show_speakers ? speakerTurns : undefined,
      }, null, 2),
    }],
  };
});

Partial vs final results

Each Result in the stream has a ResultId that persists across revisions. The streaming service sends multiple events with the same ResultId: first with IsPartial: true (the current best guess), then — when it has seen enough future audio context to finalize the utterance — a last event with IsPartial: false. This means you should key your accumulator on ResultId, replacing the entry each time you see the same ID, and only emit when IsPartial is false:

// Accumulator pattern that handles revision correctly
const pendingResults = new Map<string, string>(); // resultId → transcript

for await (const event of resultStream as AsyncIterable<any>) {
  if (!event.TranscriptEvent) continue;
  for (const result of event.TranscriptEvent.Transcript.Results ?? []) {
    const id: string = result.ResultId;
    const text: string = result.Alternatives?.[0]?.Transcript ?? '';

    if (result.IsPartial) {
      // Replace the pending draft for this result
      pendingResults.set(id, text);
    } else {
      // Finalized — emit and remove from pending map
      pendingResults.delete(id);
      console.log('FINAL:', text);
    }
  }
}

Partial results are useful for UI display (live captions) but should not be used for downstream processing — they can change character-by-character as the model hears more audio context. Final results are stable and safe to store.

Partial results stabilization

When EnablePartialResultsStabilization is true, stable words within a partial result are marked with a Stable: true flag on each Item. You can use these stable words to begin downstream processing before the full utterance finalizes — for example, triggering an intent classifier on the first stable words of a turn. Final results (IsPartial: false) never have an IsPartial flag on individual items — all items in a final result are authoritative.

Audio format requirements

// Convert browser MediaRecorder output (ogg-opus) for Transcribe Streaming
// MediaRecorder typically produces ogg-opus at 48000 Hz

const command = new StartStreamingTranscriptionCommand({
  LanguageCode: 'en-US',
  MediaEncoding: 'ogg-opus',
  MediaSampleRateHertz: 48000, // must match the recorder's actual sample rate
  AudioStream: browserAudioStream(),
  EnablePartialResultsStabilization: true,
  PartialResultsStability: 'medium',
});

Vocabulary filter and custom vocabulary in streaming

// Use an existing custom vocabulary and vocabulary filter in a streaming job
const command = new StartStreamingTranscriptionCommand({
  LanguageCode: 'en-US',
  MediaEncoding: 'pcm',
  MediaSampleRateHertz: 16000,
  AudioStream: audioStream(),
  VocabularyName: 'my-domain-vocabulary',       // must already be in READY state
  VocabularyFilterName: 'profanity-filter',       // created with CreateVocabularyFilterCommand
  VocabularyFilterMethod: 'mask',                 // 'mask' | 'remove' | 'tag'
});

The VocabularyName and VocabularyFilterName must already be in READY state before the streaming session starts. Creating or updating a vocabulary and immediately starting a stream with it may return a BadRequestException if the vocabulary has not finished compiling. Add a readiness check in your initialization path — see the custom vocabulary guide for the vocabulary wait-state pattern.

Error handling and stream teardown

try {
  const { TranscriptResultStream: resultStream } =
    await streamingClient.send(command);

  for await (const event of resultStream as AsyncIterable<any>) {
    if (event.BadRequestException) {
      // Wrong MediaSampleRateHertz, unsupported language for streaming, etc.
      throw new Error(`BadRequestException: ${event.BadRequestException.Message}`);
    }
    if (event.LimitExceededException) {
      // Concurrent streaming sessions limit reached (5 per region by default)
      throw new Error('Streaming concurrency limit exceeded — retry with backoff');
    }
    if (event.InternalFailureException) {
      // Transient service error — entire session must restart
      throw new Error(`InternalFailureException: ${event.InternalFailureException.Message}`);
    }
    if (event.ConflictException) {
      // Another stream with the same session ID is still open (rare)
      throw new Error(`ConflictException: ${event.ConflictException.Message}`);
    }
    if (!event.TranscriptEvent) continue;

    // ... process TranscriptEvent
  }
} catch (err) {
  // The HTTP/2 stream is automatically closed on exception — no explicit teardown needed
  throw err;
}

AWS Transcribe Streaming has a default concurrent session limit of 5 per region. For MCP servers under load, request a limit increase via Service Quotas before deploying. Each session persists until the audio stream generator is exhausted or the for-await loop exits — the HTTP/2 stream is automatically closed. Do not leave idle streams open: they count against the concurrency quota even if no audio is being sent.