AWS Bedrock Advanced Patterns for MCP Servers: Prompt Caching, Embeddings, Batch Inference, and Multimodal

Pattern 1: Prompt Caching — eliminating repeated system-prompt costs in MCP tool handlers

The most expensive inefficiency in AI-native MCP servers is not the output tokens — it is the input tokens consumed by re-processing identical system prompts on every tool invocation. When an MCP server calls Claude 3.5 Sonnet with a 3,000-token system prompt containing tool documentation, behavioral instructions, and reference tables, every call pays the full input price for those tokens even though the model could, in principle, reuse the KV-cache computed for the same content on the previous request. Bedrock prompt caching makes that reuse explicit via the cachePoint block: a control directive inserted into the message array that tells Bedrock "everything before this marker is stable; store the intermediate computation and reuse it on the next request that presents the same prefix."

The cachePoint block and where to place it

A cachePoint is a typed content block with a single property: { cachePoint: { type: 'default' } }. It does not produce any output text — it is a routing instruction consumed before inference begins. Two placement positions are valid:

• System array placement: Add the cachePoint as the last element in the system array after all system text blocks. Everything in the system array before the marker is checkpointed. This is the standard placement for MCP servers that have a large, stable system prompt.
• User message content array placement: Add the cachePoint as the last element of the content array within a user role message. Useful for caching few-shot examples passed as user turns, or for caching accumulated multi-turn conversation history.

The SDK type for a cachePoint in the system array is ContentBlockMemberCachePoint, which requires a type assertion in TypeScript because the union type definition may not include it in older SDK versions:

import {
  BedrockRuntimeClient,
  ConverseCommand,
  type SystemContentBlock,
} from '@aws-sdk/client-bedrock-runtime';

// Build stable system prompt once at server startup
const TOOL_DOCS = buildToolDocumentation(); // ~2500 tokens of stable reference text

const CACHED_SYSTEM: SystemContentBlock[] = [
  {
    text: `You are an MCP server diagnostics expert.
When analyzing issues, check the specific MCP error code (-32603, -32601, etc.)
and verify endpoint health via AliveMCP status page first.

Reference documentation:
${TOOL_DOCS}
`,
  },
  // cachePoint checkpoints everything above — must be last element
  { cachePoint: { type: 'default' } } as unknown as SystemContentBlock,
];

// Reuse CACHED_SYSTEM across all requests — never re-build per request
async function callBedrock(userQuery: string) {
  const response = await new BedrockRuntimeClient({ region: 'us-east-1' }).send(
    new ConverseCommand({
      modelId: 'us.anthropic.claude-3-5-sonnet-20241022-v2:0',
      system: CACHED_SYSTEM,
      messages: [{ role: 'user', content: [{ text: userQuery }] }],
      inferenceConfig: { maxTokens: 2048 },
    }),
  );

  const { cacheWriteInputTokenCount, cacheReadInputTokenCount } = response.usage ?? {};
  console.log({ cacheWriteInputTokenCount, cacheReadInputTokenCount });

  return response;
}

Token thresholds and why they silently suppress caching

Bedrock enforces a minimum token count before the cachePoint. If the token count before the marker is below the minimum, caching does not activate — no error is raised, cacheWriteInputTokenCount will be 0, and the request processes normally at standard pricing. There is no warning in the response.

If your system prompt is below 2,048 tokens on Sonnet and you want caching to activate, add substantive content: tool schema documentation, behavioral examples, reference tables, or explanatory prose about the tools. Do not pad with filler text — the quality of the system prompt degrades. The right approach is to make the system prompt genuinely useful and naturally long enough to cross the threshold.

The TTL, AZ distribution, and deployment-boundary behavior

Cache entries expire 5 minutes after their last read. Each cache read resets the 5-minute clock. For MCP servers receiving more than one tool call per 5 minutes — any production service with meaningful traffic — the cache entry stays warm indefinitely. Three operational nuances matter for production deployments:

• AZ-level granularity: Cache entries are per-availability-zone. A multi-AZ Lambda deployment will see each AZ pay a cache-write cost on its first request, then cache-read prices within the AZ. This is expected behavior — high-traffic services will establish AZ-level caches within seconds of the first request to each AZ.
• Low-traffic tools: An MCP tool called once every 10 minutes will pay cache-write cost (125%) on every call, with zero benefit. Cache only pays off when call frequency exceeds one call per 5-minute window per cachePoint. For low-traffic tools, remove the cachePoint to avoid the 25% write-cost overhead with no benefit.
• Rolling deployments: Any change to the system prompt text invalidates the cache. During a rolling restart where new instances build their cache entry, there is a brief period of increased write costs. The system prompt should be versioned alongside the code — changing it incidentally (e.g., by embedding a deployment timestamp) is a common mistake that prevents caching from working.

Troubleshooting: why cache hits are not occurring

Pattern 2: Embeddings — semantic search and RAG with Titan V2 and Cohere Embed 3

MCP servers that need to find semantically similar content — matching a user query to the right tool, retrieving relevant documentation chunks for a RAG pipeline, clustering MCP server capability descriptions, or identifying which monitored endpoint most closely matches a user's search term — need to convert text into dense vector representations. Bedrock provides two production-grade embedding model families with different trade-offs: Titan Text Embeddings V2 for symmetric search on long documents, and Cohere Embed 3 for asymmetric search with storage compression.

Titan Text Embeddings V2: symmetric search with long inputs

Titan V2 (amazon.titan-embed-text-v2:0) is called via InvokeModelCommand with a JSON body containing inputText, dimensions (256, 512, or 1024), and normalize. The 1,024-dimension model provides the highest retrieval quality and is the right default for new projects. The normalize=true parameter returns unit-length vectors, which means cosine similarity between two vectors reduces to a plain dot product — no magnitude division needed.

import {
  BedrockRuntimeClient,
  InvokeModelCommand,
} from '@aws-sdk/client-bedrock-runtime';

interface TitanEmbedResponse {
  embedding: number[];
  inputTextTokenCount: number;
}

async function embedWithTitan(
  text: string,
  dimensions: 256 | 512 | 1024 = 1024,
): Promise {
  const client = new BedrockRuntimeClient({ region: 'us-east-1' });

  const body = JSON.stringify({ inputText: text, dimensions, normalize: true });
  const response = await client.send(new InvokeModelCommand({
    modelId: 'amazon.titan-embed-text-v2:0',
    contentType: 'application/json',
    accept: 'application/json',
    body: new TextEncoder().encode(body),
  }));

  const decoded = new TextDecoder().decode(response.body);
  const result: TitanEmbedResponse = JSON.parse(decoded);
  return result.embedding;
}

// With normalize=true, cosine similarity = dot product of the two vectors
function cosineSimilarity(a: number[], b: number[]): number {
  return a.reduce((sum, val, i) => sum + val * b[i], 0);
}

Titan V2's 8,192-token limit (roughly 32,000 characters) is rarely a constraint for MCP tool inputs, but matters when embedding full document contents for a RAG pipeline. For documents longer than 8,192 tokens, chunking is required.

Cohere Embed 3: asymmetric search and storage compression

The defining characteristic of Cohere Embed 3 is asymmetric embeddings: the model was trained with separate encoder heads for short queries and longer documents. A query embedding for "what is the AliveMCP ping interval?" is numerically different from a document embedding for the same text, and when searching against documents indexed with input_type: 'search_document', the query encoded with input_type: 'search_query' retrieves more relevant results than symmetric encoding of either type. Using the same input_type for both queries and documents — the most common mistake — degrades retrieval quality by 5–15% on most RAG benchmarks.

// Corpus indexing — always search_document
async function buildSearchIndex(documents: string[]): Promise {
  const response = await client.send(new InvokeModelCommand({
    modelId: 'cohere.embed-english-v3',
    contentType: 'application/json',
    accept: 'application/json',
    body: new TextEncoder().encode(JSON.stringify({
      texts: documents,               // up to 96 texts per call
      input_type: 'search_document',  // ALWAYS for corpus indexing
      embedding_types: ['float'],
      truncate: 'END',                // safer than NONE (no error on overlong input)
    })),
  }));
  const result = JSON.parse(new TextDecoder().decode(response.body));
  return result.embeddings.float;
}

// Query embedding — always search_query
async function embedQuery(query: string): Promise {
  const response = await client.send(new InvokeModelCommand({
    modelId: 'cohere.embed-english-v3',
    contentType: 'application/json',
    accept: 'application/json',
    body: new TextEncoder().encode(JSON.stringify({
      texts: [query],
      input_type: 'search_query',  // ALWAYS for user queries
      embedding_types: ['float'],
    })),
  }));
  const result = JSON.parse(new TextDecoder().decode(response.body));
  return result.embeddings.float[0];
}

Cohere also supports quantized output: embedding_types: ['int8'] reduces storage by 4× with approximately 1% quality loss; embedding_types: ['binary'] reduces storage by 32× with 5–10% quality loss. For AliveMCP's use case of indexing ~100k MCP server capability descriptions at 1024 dimensions, int8 quantization reduces the embedding store from 400MB to 100MB with negligible quality impact.

Model selection guide

Storing and querying vectors with pgvector

Production MCP servers store embedding vectors in a searchable vector database. For servers already running PostgreSQL (common on AWS RDS or Aurora), pgvector is the lowest-friction choice — it adds a vector column type and HNSW approximate nearest-neighbor index to standard PostgreSQL without requiring a separate service.

-- pgvector schema for 1024-dim Titan embeddings
CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE mcp_server_embeddings (
  id          TEXT PRIMARY KEY,
  slug        TEXT NOT NULL,
  description TEXT NOT NULL,
  embedding   vector(1024) NOT NULL,
  indexed_at  TIMESTAMP DEFAULT NOW()
);

-- HNSW index: ef_construction=64, m=16 are good defaults for 1024-dim
CREATE INDEX ON mcp_server_embeddings
  USING hnsw (embedding vector_cosine_ops)
  WITH (ef_construction = 64, m = 16);

-- Query: 5 most similar MCP servers to a query vector
SELECT slug, description,
       1 - (embedding <=> $1::vector) AS similarity
FROM   mcp_server_embeddings
ORDER BY embedding <=> $1::vector   -- <=> = cosine distance; lower = more similar
LIMIT  5;

Note the <=> operator is cosine distance (0 = identical, 2 = maximally different), so ORDER BY ... ASC returns the most similar items first. To convert to similarity (0–1 scale), use 1 - (embedding <=> $1::vector) as shown.

Pattern 3: Batch Inference — async bulk analysis at 50% cost via CreateModelInvocationJob

The weekly health summary that processes 10,000 monitored endpoint logs to identify deterioration patterns, the nightly classification pass that categorizes new MCP servers by capability cluster, the monthly SLA report that generates a narrative summary for each enterprise client's private endpoints — these workloads have two characteristics in common: they process many records in one go, and the user is not waiting for a result in real time. Bedrock batch inference is architecturally correct for exactly this pattern: a JSONL input file uploaded to S3, a single CreateModelInvocationJobCommand call, and results delivered to an S3 output prefix when the job completes — at 50% of on-demand pricing.

Input JSONL format and the model-native body requirement

The input file is a JSONL file where each line is a JSON object with two fields: recordId (a unique identifier for this record) and modelInput (the raw model request body). The critical nuance: modelInput uses the model's native request format — for Claude models accessed via Bedrock, this is the Anthropic Messages API format with anthropic_version: 'bedrock-2023-05-31', not the Converse API format. Sending a Converse API body as modelInput produces a ValidationException for every record.

// Correct: Anthropic Messages API format for Claude via Bedrock batch
const record = {
  recordId: `endpoint-${serverSlug}`,
  modelInput: {
    anthropic_version: 'bedrock-2023-05-31',  // required for Claude via Bedrock batch
    max_tokens: 1024,
    messages: [{ role: 'user', content: errorLogText }],
    system: 'You are an MCP server diagnostics expert. Summarize the root cause in one sentence.',
  },
};

// Incorrect: Converse API format will be rejected
// modelInput: { messages: [...], inferenceConfig: {...} }  // ValidationException

IAM role with bedrock.amazonaws.com trust

Batch inference requires a dedicated IAM role — not the Lambda execution role — that Bedrock can assume. The trust policy must name bedrock.amazonaws.com as the service principal and should include a SourceAccount condition to prevent confused deputy attacks.

// Trust policy (applied to the batch inference IAM role)
{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": { "Service": "bedrock.amazonaws.com" },
    "Action": "sts:AssumeRole",
    "Condition": {
      "StringEquals": { "aws:SourceAccount": "123456789012" },
      "ArnLike": { "aws:SourceArn": "arn:aws:bedrock:us-east-1:123456789012:*" }
    }
  }]
}

// Permission policy (also attached to the batch inference IAM role)
{
  "Statement": [
    { "Action": ["bedrock:InvokeModel"], "Resource": ["arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-5-sonnet-20241022-v2:0"] },
    { "Action": ["s3:GetObject", "s3:ListBucket"], "Resource": ["arn:aws:s3:::my-batch-bucket", "arn:aws:s3:::my-batch-bucket/*"] },
    { "Action": ["s3:PutObject"], "Resource": ["arn:aws:s3:::my-batch-output-bucket/*"] }
  ]
}

Job creation, polling, and EventBridge notification

An MCP tool that submits a batch job should return the job ARN immediately — never block waiting for completion. The caller gets a handle they can use to poll status or receive a notification. For background pipelines, EventBridge is preferable to polling because it requires no background task management:

import { BedrockClient, CreateModelInvocationJobCommand } from '@aws-sdk/client-bedrock';

const bedrock = new BedrockClient({ region: 'us-east-1' });

async function submitBatchJob(inputS3Uri: string, outputS3Uri: string): Promise {
  const response = await bedrock.send(new CreateModelInvocationJobCommand({
    modelId: 'anthropic.claude-3-5-sonnet-20241022-v2:0',
    jobName: `mcp-analysis-${Date.now()}`,
    roleArn: process.env.BEDROCK_BATCH_ROLE_ARN!,
    inputDataConfig: {
      s3InputDataConfig: { s3Uri: inputS3Uri, s3InputFormat: 'JSONLines' },
    },
    outputDataConfig: {
      s3OutputDataConfig: { s3Uri: outputS3Uri },
    },
  }));
  return response.jobArn!;  // store this — it is the durable job identifier
}

// EventBridge rule (set up via CDK or Console — not runtime code):
// {
//   "source": ["aws.bedrock"],
//   "detail-type": ["Batch Inference Job State Change"],
//   "detail": { "status": ["Completed", "Failed"] }
// }
// Target: Lambda that reads output S3 and processes results

Output processing and inline error records

The output JSONL mirrors the input: each line contains a recordId plus either a modelOutput (success) or an error object (failure). The job-level status reports Completed even when some records failed — a completed job with 500 error records out of 10,000 is still Completed, not Failed. Always process the failure count before declaring the job a success.

interface BatchOutputRecord {
  recordId: string;
  modelOutput?: { content: Array<{ type: 'text'; text: string }>; stop_reason: string };
  error?: { errorCode: string; errorMessage: string };
}

async function processOutput(outputS3Uri: string) {
  // List .jsonl files under the output prefix and read each one
  const records: BatchOutputRecord[] = await readOutputJsonl(outputS3Uri);

  const successes = records.filter(r => r.modelOutput);
  const failures  = records.filter(r => r.error);

  console.log(`Results: ${successes.length} succeeded, ${failures.length} failed`);
  // Handle failures: ThrottledException → retry via on-demand
  //                  ModelTimeoutException → reduce max_tokens or simplify prompt
  //                  ValidationException   → fix malformed modelInput records
  for (const f of failures) {
    console.error(`Record ${f.recordId}: ${f.error!.errorCode} — ${f.error!.errorMessage}`);
  }
}

Supported models and hard limits

Hard limits: 50,000 records per job; 1 concurrent running job per account per region (additional jobs queue — they do not fail); jobs expire after 7 days if not completed. Set a CloudWatch Alarm on the EventBridge Failed job state to get notified when a batch job fails entirely (distinct from records failing within a completed job).

Pattern 4: Multimodal — ImageBlock and DocumentBlock in the Converse API

MCP servers that accept screenshots of monitoring dashboards, infrastructure diagrams, error screenshots, PDF architecture documents, or Excel health reports need the Bedrock Converse API's multimodal content blocks. The implementation looks simple — add an ImageBlock or DocumentBlock to the user message content array — but the model support matrix is restrictive enough that sending the wrong content type to the wrong model produces a silent runtime error rather than a type error at compile time, and the image token pricing model has a multi-turn cost accumulation pattern that can multiply expected costs by 10× on long conversations.

Model support matrix

The safe default for production multimodal MCP tools is Claude 3.5 Sonnet or Claude Sonnet 4.x — full image support, all document formats, and the strongest multimodal reasoning. Use Claude 3.5 Haiku for high-volume image tasks where latency and cost matter more than reasoning depth. Use Amazon Nova Lite for very high-volume, cost-sensitive image classification tasks (Nova's megapixel pricing is significantly cheaper than Claude's token-based pricing for small images).

Sending images via inline bytes and S3 source

Inline bytes is the simplest approach: read the image into memory, wrap it in an ImageBlock, and include it in the user message content array. The SDK automatically base64-encodes the Uint8Array.

import { readFileSync } from 'fs';
import { BedrockRuntimeClient, ConverseCommand, type ContentBlock } from '@aws-sdk/client-bedrock-runtime';

type ImageFormat = 'jpeg' | 'png' | 'gif' | 'webp';

async function analyzeImage(imagePath: string, format: ImageFormat, prompt: string): Promise {
  const imageBytes = readFileSync(imagePath);
  if (imageBytes.length > 5 * 1024 * 1024) throw new Error('Image exceeds 5MB limit');

  const client = new BedrockRuntimeClient({ region: 'us-east-1' });
  const response = await client.send(new ConverseCommand({
    modelId: 'us.anthropic.claude-3-5-sonnet-20241022-v2:0',
    messages: [{
      role: 'user',
      content: [
        { image: { format, source: { bytes: imageBytes } } } as ContentBlock,
        { text: prompt },
      ],
    }],
    inferenceConfig: { maxTokens: 2048 },
  }));

  return response.output?.message?.content
    ?.filter(b => 'text' in b)
    .map(b => (b as { text: string }).text)
    .join('') ?? '';
}

// S3 source: Bedrock fetches the image — caller role needs s3:GetObject
async function analyzeImageFromS3(s3Uri: string, format: ImageFormat, prompt: string): Promise {
  const client = new BedrockRuntimeClient({ region: 'us-east-1' });
  const response = await client.send(new ConverseCommand({
    modelId: 'us.anthropic.claude-3-5-sonnet-20241022-v2:0',
    messages: [{
      role: 'user',
      content: [
        { image: { format, source: { s3Location: { uri: s3Uri, bucketOwner: process.env.AWS_ACCOUNT_ID } } } } as ContentBlock,
        { text: prompt },
      ],
    }],
    inferenceConfig: { maxTokens: 1024 },
  }));

  return response.output?.message?.content
    ?.filter(b => 'text' in b)
    .map(b => (b as { text: string }).text)
    .join('') ?? '';
}

Document processing: PDF, DOCX, XLSX, CSV

DocumentBlock handles non-image documents. Bedrock extracts text and structure from these formats and presents them to the model as semantic content — not as pixels — so the model can reason about table structure in Excel files, paragraph flow in Word documents, and section hierarchy in PDFs without needing vision capabilities.

type DocumentFormat = 'pdf' | 'docx' | 'xlsx' | 'csv' | 'txt' | 'md' | 'html';

async function analyzeDocument(docPath: string, format: DocumentFormat, question: string): Promise {
  const docBytes = readFileSync(docPath);
  if (docBytes.length > 4.5 * 1024 * 1024) throw new Error('Document exceeds 4.5MB limit');

  const client = new BedrockRuntimeClient({ region: 'us-east-1' });
  const response = await client.send(new ConverseCommand({
    modelId: 'us.anthropic.claude-3-5-sonnet-20241022-v2:0',  // Claude 3.5+ for DOCX/XLSX/CSV
    messages: [{
      role: 'user',
      content: [
        {
          document: {
            format,
            name: docPath.split('/').pop()!,  // filename for model context
            source: { bytes: docBytes },
          },
        } as ContentBlock,
        { text: question },
      ],
    }],
    inferenceConfig: { maxTokens: 4096 },
  }));

  return response.output?.message?.content
    ?.filter(b => 'text' in b)
    .map(b => (b as { text: string }).text)
    .join('') ?? '';
}

Image token pricing and the multi-turn cost trap

Image tokens are priced as input tokens. The approximation for Claude is (width × height) / 750 after automatic downsampling (Claude resizes images whose longest edge exceeds 1,568 pixels). At Claude 3.5 Sonnet pricing ($3/M input tokens):

The multi-turn cost trap: when an image appears in turn 1 of a multi-turn conversation, it remains in the context window for all subsequent turns. The model re-processes the image tokens on every call. A 10-turn conversation about a single 1024×1024 screenshot pays approximately 1,398 image tokens × 10 turns = 13,980 image tokens, compared to 1,398 tokens for a single-turn query. The correct mitigation: in turn 1, generate a detailed text description of the image and discard the image from subsequent turns.

// Anti-pattern: image re-billed on every turn
const messages: Message[] = [
  { role: 'user', content: [imageBlock, { text: 'Describe this dashboard.' }] },
  { role: 'assistant', content: [{ text: assistantResponse1 }] },
  // image tokens charged again here ↓
  { role: 'user', content: [{ text: 'What caused the spike at 3pm?' }] },
];

// Correct: describe image in turn 1, drop it from subsequent turns
const description = await analyzeImage(imagePath, 'png', 'Describe every element of this monitoring dashboard in detail.');
const followUpMessages: Message[] = [
  { role: 'user', content: [{ text: `Dashboard description: ${description}\n\nWhat caused the spike at 3pm?` }] },
  // no image in the message history — no image token re-billing
];

Common errors and solutions

Production checklist: Bedrock advanced patterns for MCP servers