MCP Server Bedrock InvokeModel vs Converse — payload formats, streaming, model families

Converse vs InvokeModel — decision matrix

The choice between Converse and InvokeModel is not primarily about performance — both APIs route to the same underlying model endpoints, and end-to-end latency is comparable for equivalent workloads. The decision is about API surface area and capability access.

ConverseCommand offers a unified messages array with typed content blocks (text, image, document, toolUse, toolResult), a normalized InferenceConfiguration object, and a toolConfig field for structured tool calling without model-specific prompt engineering. The same handler code runs against anthropic.claude-3-5-sonnet-20241022-v2:0, meta.llama3-70b-instruct-v1:0, and amazon.titan-text-premier-v1:0 without modification. Converse also provides converseStream via ConverseStreamCommand, which returns a typed async iterable of discriminated union events rather than raw bytes.

InvokeModelCommand sends a raw JSON body serialized to Uint8Array via TextEncoder and receives a raw byte response. Every model family has a different request schema and a different response schema. The upside is complete access to model-specific parameters: Claude's thinking block with budget_tokens for extended reasoning, Titan's textGenerationConfig.returnLikelihoods for log-probability analysis, Llama's raw prompt format with <|begin_of_text|> special tokens. If your MCP tool is purpose-built for one model and needs those parameters, InvokeModel is the correct choice.

For streaming, the Converse path uses ConverseStreamCommand which returns typed events (messageStart, contentBlockStart, contentBlockDelta, messageStop). The InvokeModel path uses InvokeModelWithResponseStreamCommand, which returns an async iterable of raw PayloadPart events that you must decode and parse yourself — and whose JSON boundaries do not align with chunk boundaries.

// Decision matrix as code — pick based on your requirements:
type ApiChoice =
  | 'converse'           // multi-model, tool use, streaming, most MCP tools
  | 'invoke-model';      // model-specific params, extended thinking, Titan embeddings

function chooseApi(modelId: string, options: {
  needsToolUse?: boolean;
  needsExtendedThinking?: boolean;
  needsTitanEmbedDimension?: boolean;
  multiModel?: boolean;
}): ApiChoice {
  if (options.needsToolUse) return 'converse';          // ConverseToolConfig
  if (options.multiModel) return 'converse';            // portable across families
  if (options.needsExtendedThinking && modelId.startsWith('anthropic.claude')) {
    return 'invoke-model';  // budget_tokens not in Converse InferenceConfiguration
  }
  if (options.needsTitanEmbedDimension && modelId.startsWith('amazon.titan')) {
    return 'invoke-model';  // outputEmbeddingLength not in Converse
  }
  // Default: Converse for simplicity
  return 'converse';
}

Model-specific InvokeModel payload formats

Each model family on Bedrock has its own request and response JSON schema. The schemas are stable within a model family across model versions — a Claude 3.5 Sonnet request body works for Claude 3 Opus with the same top-level structure — but differ significantly across families.

For Claude (anthropic.claude-*), the request body requires anthropic_version set to the literal string "bedrock-2023-05-31" (this version string is fixed for all Claude on Bedrock calls regardless of the model version), max_tokens (required, integer), and messages (array of {role, content} objects). The optional system field is a string, not an array. The response body has a content array of typed blocks and a stop_reason field.

For Titan Text (amazon.titan-text-*), the request wraps the prompt in inputText (a plain string — no messages array) and accepts configuration under textGenerationConfig with fields maxTokenCount (not max_tokens), temperature, topP, and stopSequences. The response has a results array where each entry has outputText and completionReason.

For Meta Llama 3 (meta.llama3-*), the request uses a prompt field containing the raw text with Llama 3's special tokens baked in: <|begin_of_text|>, <|start_header_id|>system<|end_header_id|>, <|eot_id|>, etc. There is no structured messages array — you construct the prompt string manually. Configuration fields are max_gen_len (not max_tokens), temperature, and top_p. The response has a generation field with the raw completion text.

import {
  BedrockRuntimeClient,
  InvokeModelCommand,
  type InvokeModelCommandInput,
} from '@aws-sdk/client-bedrock-runtime';

const client = new BedrockRuntimeClient({ region: process.env.AWS_REGION ?? 'us-east-1' });
const enc = new TextEncoder();
const dec = new TextDecoder();

// --- Claude (anthropic.claude-*) ---
async function invokeModelClaude(
  modelId: string,
  systemPrompt: string,
  userMessage: string,
  maxTokens = 1024,
): Promise<string> {
  const body = {
    anthropic_version: 'bedrock-2023-05-31',  // always this exact string for Bedrock
    max_tokens: maxTokens,
    system: systemPrompt,
    messages: [{ role: 'user', content: userMessage }],
    temperature: 0.7,
    top_p: 0.9,
  };

  const input: InvokeModelCommandInput = {
    modelId,
    contentType: 'application/json',
    accept: 'application/json',
    body: enc.encode(JSON.stringify(body)),
  };

  const response = await client.send(new InvokeModelCommand(input));
  const parsed = JSON.parse(dec.decode(response.body));
  // parsed.content[0].text for text generation
  // parsed.stop_reason: "end_turn" | "max_tokens" | "stop_sequence"
  return parsed.content[0]?.text ?? '';
}

// --- Titan Text (amazon.titan-text-*) ---
async function invokeModelTitan(
  modelId: string, // e.g. "amazon.titan-text-premier-v1:0"
  prompt: string,
  maxTokenCount = 512,
): Promise<string> {
  const body = {
    inputText: prompt,
    textGenerationConfig: {
      maxTokenCount,   // note: maxTokenCount, not max_tokens
      temperature: 0.7,
      topP: 0.9,
      stopSequences: [],
    },
  };

  const response = await client.send(new InvokeModelCommand({
    modelId,
    contentType: 'application/json',
    accept: 'application/json',
    body: enc.encode(JSON.stringify(body)),
  }));

  const parsed = JSON.parse(dec.decode(response.body));
  // parsed.results[0].outputText
  // parsed.results[0].completionReason: "FINISH" | "LENGTH" | "CONTENT_FILTERED"
  return parsed.results?.[0]?.outputText ?? '';
}

// --- Meta Llama 3 (meta.llama3-*) ---
async function invokeModelLlama(
  modelId: string, // e.g. "meta.llama3-70b-instruct-v1:0"
  systemPrompt: string,
  userMessage: string,
  maxGenLen = 512,
): Promise<string> {
  // Llama 3 requires raw prompt with special tokens — no messages array
  const prompt = [
    '<|begin_of_text|>',
    '<|start_header_id|>system<|end_header_id|>',
    `\n${systemPrompt}\n`,
    '<|eot_id|>',
    '<|start_header_id|>user<|end_header_id|>',
    `\n${userMessage}\n`,
    '<|eot_id|>',
    '<|start_header_id|>assistant<|end_header_id|>',
  ].join('');

  const body = {
    prompt,
    max_gen_len: maxGenLen,  // note: max_gen_len, not max_tokens
    temperature: 0.7,
    top_p: 0.9,
  };

  const response = await client.send(new InvokeModelCommand({
    modelId,
    contentType: 'application/json',
    accept: 'application/json',
    body: enc.encode(JSON.stringify(body)),
  }));

  const parsed = JSON.parse(dec.decode(response.body));
  // parsed.generation — the raw completion text
  // parsed.stop_reason: "stop" | "length"
  return parsed.generation ?? '';
}

Streaming with InvokeModelWithResponseStream — chunk reassembly

The InvokeModelWithResponseStreamCommand returns a response where body is an async iterable of discriminated union events. The only event type you need to handle for generation is PayloadPart: { chunk: { bytes: Uint8Array } }. Each bytes value is a partial piece of the model's response — and critically, JSON object boundaries do not align with chunk boundaries.

For Claude, each chunk that arrives is itself a complete JSON object describing a streaming event: {"type": "content_block_delta", "index": 0, "delta": {"type": "text_delta", "text": "Hello"}}. Claude's streaming format wraps each server-sent event in its own JSON envelope, so you parse each chunk independently after decoding. Event types to handle are: message_start (contains initial usage metadata), content_block_start, content_block_delta (contains incremental text in the delta), content_block_stop, and message_stop (contains final stop_reason).

For Titan Text, the entire response arrives as a single chunk — there is no incremental streaming within a single chunk. The bytes decode directly to the same JSON structure as a non-streaming response. If you need word-level streaming from Titan, you must implement client-side buffering at the sentence or paragraph level, as Titan does not emit incremental token events.

The safe reassembly pattern is to maintain a buffer string, append each decoded chunk, then attempt JSON.parse(buffer) in a try/catch. If parsing succeeds, process the object and clear the buffer. This handles both Claude's per-event JSON and any model that emits partial JSON across chunk boundaries.

import {
  InvokeModelWithResponseStreamCommand,
  type InvokeModelWithResponseStreamCommandInput,
  type ResponseStream,
} from '@aws-sdk/client-bedrock-runtime';

async function* streamClaudeResponse(
  modelId: string,
  userMessage: string,
  maxTokens = 1024,
): AsyncGenerator<string, void, unknown> {
  const body = {
    anthropic_version: 'bedrock-2023-05-31',
    max_tokens: maxTokens,
    messages: [{ role: 'user', content: userMessage }],
  };

  const response = await client.send(new InvokeModelWithResponseStreamCommand({
    modelId,
    contentType: 'application/json',
    accept: 'application/json',
    body: enc.encode(JSON.stringify(body)),
  }));

  if (!response.body) return;

  let buffer = '';

  for await (const event of response.body) {
    if (!('chunk' in event) || !event.chunk?.bytes) continue;

    // Append decoded bytes to buffer
    buffer += dec.decode(event.chunk.bytes, { stream: true });

    // For Claude, each chunk IS a complete JSON event — try parse immediately
    // For models that split JSON across chunks, this loop handles reassembly
    let parsed: Record<string, unknown>;
    try {
      parsed = JSON.parse(buffer);
      buffer = ''; // successful parse — clear buffer
    } catch {
      // Incomplete JSON — wait for more chunks
      continue;
    }

    // Claude streaming event types
    if (parsed.type === 'content_block_delta') {
      const delta = parsed.delta as { type: string; text?: string };
      if (delta.type === 'text_delta' && delta.text) {
        yield delta.text;
      }
    } else if (parsed.type === 'message_stop') {
      // Final event — contains stop_reason in amazon-bedrock-invocationMetrics
      break;
    }
    // message_start: contains usage.input_tokens for cost tracking
    // content_block_start: signals start of a new content block
    // content_block_stop: signals end of current content block
  }
}

// Usage in an MCP tool with Server-Sent Events or WebSocket transport:
async function handleStreamingTool(prompt: string): Promise<string> {
  const chunks: string[] = [];
  for await (const text of streamClaudeResponse(
    'us.anthropic.claude-3-5-sonnet-20241022-v2:0',
    prompt,
  )) {
    chunks.push(text);
    // In a real MCP server, you would stream these via MCP's progress notifications
    // or accumulate and return as a complete result
  }
  return chunks.join('');
}

Extended thinking with Claude on Bedrock

Claude's extended thinking capability — where the model produces an explicit thinking block containing its chain-of-reasoning before generating a final answer — is available on Bedrock through InvokeModel only. As of 2026, the Converse API does not expose budget_tokens for Claude on Bedrock, so if your MCP tool requires extended thinking you must use the InvokeModelCommand path with the raw Claude request format.

Extended thinking is configured by adding a thinking object to the request body with two fields: type: "enabled" and budget_tokens (an integer between 1024 and 100000 inclusive). The budget_tokens value caps how many tokens Claude may spend on internal reasoning — it does not guarantee that many tokens will be used, but Claude will not exceed the budget. Higher budgets improve reasoning quality on complex multi-step problems but increase both latency and cost, since thinking tokens are billed at the standard output token rate for the model.

When thinking is enabled, the response's content array contains two types of blocks: one or more thinking blocks (each with a thinking field containing the internal reasoning text and a signature field used for cache validation) followed by one or more text blocks containing the final answer. You can log the thinking content for observability or discard it — only return the text blocks to the MCP client unless your tool explicitly surfaces reasoning to the caller.

interface ThinkingResponse {
  thinking: string;
  answer: string;
  inputTokens: number;
  outputTokens: number;
  thinkingTokensEstimate: number;
}

async function invokeClaudeWithThinking(
  userMessage: string,
  budgetTokens = 8000,  // min 1024, max 100000
  modelId = 'us.anthropic.claude-3-7-sonnet-20250219-v1:0',
): Promise<ThinkingResponse> {
  const body = {
    anthropic_version: 'bedrock-2023-05-31',
    max_tokens: budgetTokens + 4096,  // must be greater than budget_tokens
    thinking: {
      type: 'enabled',
      budget_tokens: budgetTokens,
    },
    messages: [{ role: 'user', content: userMessage }],
    // Note: temperature must be 1 when thinking is enabled (Claude requirement)
    temperature: 1,
  };

  const response = await client.send(new InvokeModelCommand({
    modelId,
    contentType: 'application/json',
    accept: 'application/json',
    body: enc.encode(JSON.stringify(body)),
  }));

  const parsed = JSON.parse(dec.decode(response.body));

  let thinkingText = '';
  let answerText = '';

  for (const block of (parsed.content ?? [])) {
    if (block.type === 'thinking') {
      thinkingText += block.thinking ?? '';
    } else if (block.type === 'text') {
      answerText += block.text ?? '';
    }
    // block.type === 'redacted_thinking' means thinking was filtered —
    // signature is present but thinking field is absent; treat as opaque
  }

  const inputTokens: number = parsed.usage?.input_tokens ?? 0;
  const outputTokens: number = parsed.usage?.output_tokens ?? 0;
  // Thinking tokens count toward output_tokens — estimate based on thinking block length
  const thinkingTokensEstimate = Math.ceil(thinkingText.length / 4);

  return { thinking: thinkingText, answer: answerText, inputTokens, outputTokens, thinkingTokensEstimate };
}

MCP tool dispatcher — routing to Converse or InvokeModel by model prefix

An MCP server that supports multiple model families needs a routing layer that selects the right API and serialization format based on the model ID. The model ID prefix is the reliable discriminant: all Anthropic Claude models start with anthropic.claude or (for cross-region inference profiles) us.anthropic.claude, eu.anthropic.claude, or ap.anthropic.claude; all Titan models start with amazon.titan; all Llama models start with meta.llama; all Mistral models start with mistral..

Cache BedrockRuntimeClient instances at module scope. The SDK client handles connection pooling internally but the constructor itself initializes credential resolution, regional endpoint selection, and retry configuration — running it per-request adds 50–150 ms of overhead depending on the credential provider chain. A single module-scoped client is safe for concurrent use across multiple MCP tool invocations.

import {
  BedrockRuntimeClient,
  ConverseCommand,
  InvokeModelCommand,
  type ConverseCommandInput,
} from '@aws-sdk/client-bedrock-runtime';

// Single client instance — safe for concurrent use, handles connection pooling
const bedrockClient = new BedrockRuntimeClient({
  region: process.env.AWS_REGION ?? 'us-east-1',
  maxAttempts: 3,  // built-in retry for ThrottlingException and transient errors
});

type ModelFamily = 'claude' | 'titan' | 'llama' | 'mistral' | 'cohere' | 'unknown';

function detectModelFamily(modelId: string): ModelFamily {
  // Strip cross-region inference profile prefix: "us.", "eu.", "ap."
  const normalized = modelId.replace(/^(us|eu|ap)\./, '');

  if (normalized.startsWith('anthropic.claude')) return 'claude';
  if (normalized.startsWith('amazon.titan')) return 'titan';
  if (normalized.startsWith('meta.llama')) return 'llama';
  if (normalized.startsWith('mistral.')) return 'mistral';
  if (normalized.startsWith('cohere.')) return 'cohere';
  return 'unknown';
}

interface InvokeParams {
  modelId: string;
  userMessage: string;
  systemPrompt?: string;
  maxTokens?: number;
  temperature?: number;
  // Set to trigger InvokeModel path for Claude extended thinking
  extendedThinkingBudget?: number;
}

interface InvokeResult {
  text: string;
  thinking?: string;
  inputTokens?: number;
  outputTokens?: number;
  api: 'converse' | 'invoke-model';
}

async function invokeModel(params: InvokeParams): Promise<InvokeResult> {
  const { modelId, userMessage, systemPrompt, maxTokens = 1024, temperature = 0.7 } = params;
  const family = detectModelFamily(modelId);

  // Extended thinking requires InvokeModel for Claude
  if (params.extendedThinkingBudget && family === 'claude') {
    const result = await invokeClaudeWithThinking(userMessage, params.extendedThinkingBudget, modelId);
    return {
      text: result.answer,
      thinking: result.thinking,
      inputTokens: result.inputTokens,
      outputTokens: result.outputTokens,
      api: 'invoke-model',
    };
  }

  // Titan and Llama have model-specific quirks — use InvokeModel
  if (family === 'titan') {
    const text = await invokeModelTitan(modelId, userMessage, maxTokens);
    return { text, api: 'invoke-model' };
  }

  if (family === 'llama') {
    const text = await invokeModelLlama(modelId, systemPrompt ?? '', userMessage, maxTokens);
    return { text, api: 'invoke-model' };
  }

  // Default: Converse for Claude (without thinking), Mistral, Cohere, and unknown models
  const converseInput: ConverseCommandInput = {
    modelId,
    messages: [{ role: 'user', content: [{ text: userMessage }] }],
    ...(systemPrompt ? { system: [{ text: systemPrompt }] } : {}),
    inferenceConfig: { maxTokens, temperature },
  };

  const response = await bedrockClient.send(new ConverseCommand(converseInput));

  const text = response.output?.message?.content
    ?.filter((b): b is { text: string } => 'text' in b)
    .map(b => b.text)
    .join('') ?? '';

  return {
    text,
    inputTokens: response.usage?.inputTokens,
    outputTokens: response.usage?.outputTokens,
    api: 'converse',
  };
}

// MCP tool registration using the dispatcher:
import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
import { z } from 'zod';

const server = new McpServer({ name: 'bedrock-multi-model', version: '1.0.0' });

server.tool(
  'bedrock_invoke',
  'Invoke any Bedrock foundation model with automatic API routing',
  {
    modelId: z.string().min(1),
    prompt: z.string().min(1).max(50000),
    systemPrompt: z.string().optional(),
    maxTokens: z.number().int().min(1).max(128000).optional(),
    extendedThinking: z.boolean().optional(),
  },
  async ({ modelId, prompt, systemPrompt, maxTokens, extendedThinking }) => {
    const result = await invokeModel({
      modelId,
      userMessage: prompt,
      systemPrompt,
      maxTokens,
      extendedThinkingBudget: extendedThinking ? 8000 : undefined,
    });

    const content: Array<{ type: 'text'; text: string }> = [];
    if (result.thinking) {
      content.push({ type: 'text', text: `[Thinking]\n${result.thinking}\n\n[Response]` });
    }
    content.push({ type: 'text', text: result.text });

    return { content };
  },
);

Common failure modes