AWS Polly SSML Markup in MCP Servers — Prosody, Phoneme, Say-As, Pauses

SSML tool implementation

import {
  PollyClient,
  SynthesizeSpeechCommand,
  OutputFormat,
  Engine,
  TextType,
  VoiceId,
} from '@aws-sdk/client-polly';
import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
import { z } from 'zod';

const polly = new PollyClient({ region: 'us-east-1' });
const server = new McpServer({ name: 'polly-ssml-tools', version: '1.0.0' });

// Escape XML special characters in user-supplied strings before inserting into SSML
function escapeXml(str: string): string {
  return str
    .replace(/&/g, '&')
    .replace(/</g, '&lt;')
    .replace(/>/g, '&gt;')
    .replace(/"/g, '&quot;')
    .replace(/'/g, '&apos;');
}

server.tool('synthesize_ssml', {
  ssml: z.string().max(6000).describe('Full SSML string including <speak> wrapper'),
  voice_id: z.string().default('Joanna'),
}, async ({ ssml, voice_id }) => {
  const { AudioStream } = await polly.send(new SynthesizeSpeechCommand({
    Text: ssml,
    TextType: TextType.SSML,
    VoiceId: voice_id as VoiceId,
    OutputFormat: OutputFormat.MP3,
    Engine: Engine.NEURAL,
  }));

  if (!AudioStream) throw new Error('No audio stream');
  const chunks: Uint8Array[] = [];
  for await (const chunk of AudioStream as AsyncIterable<Uint8Array>) chunks.push(chunk);
  const buf = Buffer.concat(chunks);

  return {
    content: [{
      type: 'text',
      text: JSON.stringify({ audio_base64: buf.toString('base64'), bytes: buf.byteLength }),
    }],
  };
});

Pauses with <break>

The <break> element inserts silence. It accepts either a time attribute (in seconds or milliseconds) or a strength attribute (preset relative durations):

<speak>
  First, let me summarize the key findings.
  <break time="1s"/>
  Revenue is up 12 percent year over year.
  <break time="500ms"/>
  Operating costs decreased by 8 percent.
  <break strength="x-strong"/>
  In conclusion, the business is growing efficiently.
</speak>

The time attribute accepts values up to 10 seconds (time="10s"). Values with units other than s or ms are rejected. Do not use <break> between words inside a clause to simulate a dramatic pause — use strength="none" to remove undesired commas or use <prosody rate="slow"> around the clause instead.

Prosody: rate, pitch, and volume

<speak>
  <!-- Slow down for important information -->
  Your verification code is
  <prosody rate="x-slow" volume="loud">
    <say-as interpret-as="digits">482910</say-as>
  </prosody>.
  <break time="800ms"/>
  <!-- Speed up for a disclaimer -->
  <prosody rate="fast" volume="soft">
    Terms and conditions apply. See website for details.
  </prosody>
</speak>

Prosody tags can be nested. Inner tags take precedence over outer tags for the same attribute. Keep pitch adjustments under ±10% for natural-sounding output — extreme pitch values produce an unnatural chipmunk or low-robot effect. The standard engine supports prosody more reliably than neural; the neural engine may ignore subtle rate and pitch adjustments and apply its own prosodic model instead.

Pronunciation with <phoneme>

Domain-specific terms, product names, and proper nouns are frequently mispronounced by TTS systems. Use <phoneme> with International Phonetic Alphabet (IPA) to override the default pronunciation:

<speak>
  Welcome to <phoneme alphabet="ipa" ph="ˈeɪlɪvˌɛm.siˈpiː">AliveMCP</phoneme>,
  the uptime monitoring dashboard for
  <phoneme alphabet="ipa" ph="ˌɛm.siˈpiː">MCP</phoneme> servers.
  <break time="500ms"/>
  Your server is currently
  <phoneme alphabet="ipa" ph="ˈhɛlθi">healthy</phoneme>.
</speak>

Polly also supports the X-SAMPA phoneme alphabet via alphabet="x-sampa" — useful if your team already has X-SAMPA pronunciation data. IPA is preferred for English. To find the IPA transcription for a word, use a dictionary tool or the CMU Pronouncing Dictionary. When correcting a product name, test the SSML with SynthesizeSpeechCommand in the Polly console before shipping to production — IPA transcriptions are not validated by the API and a wrong IPA string silently produces garbled audio.

Formatting with <say-as>

The <say-as interpret-as> element tells Polly how to interpret and speak a text value. Without it, Polly uses heuristics that are often wrong for structured data:

<speak>
  <!-- Read digits individually (not as a number: "four hundred and four") -->
  Error code <say-as interpret-as="digits">404</say-as>.

  <!-- Spell out the letters of an acronym -->
  Your <say-as interpret-as="spell-out">API</say-as> key has been rotated.

  <!-- Phone number with area code -->
  Call us at <say-as interpret-as="telephone">+1-800-555-0100</say-as>.

  <!-- Date: "November fourth twenty twenty-six" -->
  Created on <say-as interpret-as="date" format="mdy">11/04/2026</say-as>.

  <!-- Time: "ten thirty AM" -->
  Scheduled at <say-as interpret-as="time" format="hms12">10:30am</say-as>.

  <!-- Cardinal number -->
  You have <say-as interpret-as="cardinal">1024</say-as> unread messages.

  <!-- Ordinal -->
  This is the <say-as interpret-as="ordinal">3</say-as> attempt.
</speak>

Acronym expansion with <sub>

Use <sub alias="..."> to replace a term with an alternative pronunciation without changing the displayed text. This is useful when the spoken form of an acronym or symbol differs from its written form:

<speak>
  Your <sub alias="sequel">SQL</sub> query completed in 42 milliseconds.
  The <sub alias="Structured Query Language">SQL</sub> standard was updated in 2023.
  Response time: <sub alias="forty-two milliseconds">42ms</sub>.
  Uptime: <sub alias="ninety-nine point nine percent">99.9%</sub>.
</speak>

Unlike <phoneme> which specifies IPA, <sub alias> takes a plain-text string that Polly synthesizes normally. This makes it easier to use than IPA for most substitutions — you write what you want spoken, not a phonetic encoding. <sub> is not supported by the generative engine.

Inline language switching with <lang>

<speak>
  The French phrase for "good morning" is
  <lang xml:lang="fr-FR">bonjour</lang>.
  In Japanese, it's
  <lang xml:lang="ja-JP">おはようございます</lang>.
</speak>

The <lang> tag switches pronunciation rules for the enclosed text. The outer voice's language rules apply to all text outside the tag. This feature requires that the selected VoiceId supports the target language as an additional language — not all voices do. Query GetVoicesCommand and inspect AdditionalLanguageCodes for the voice you're using. If the voice doesn't support the target language, Polly ignores the <lang> tag and applies the voice's primary language rules.

Common SSML pitfalls