Bedrock Prompt Caching in MCP Servers — cachePoint, token thresholds, cost reduction

How prompt caching works in Bedrock

When Bedrock processes an inference request it must run the transformer's attention mechanism over every input token. For a 4000-token system prompt followed by a 50-token user query, that's 4050 tokens of full-context computation — even though the system prompt is identical to the previous request. Prompt caching stores the intermediate KV-cache state computed for the tokens before the cachePoint marker on a dedicated cache tier. A subsequent request that presents the same token prefix up to the same marker retrieves the cached state instead of recomputing it.

The cachePoint block is available in the Bedrock Converse API for Claude model families. It is not supported for Titan, Llama, Mistral, or Command R+ via Bedrock. The block is a typed union member in the content array — the SDK type for it is ContentBlockMemberCachePoint. A cachePoint does not produce any output text; it is a control directive consumed by the Bedrock routing layer.

There are two positions where a cachePoint is valid: inside the system array (after the last system text block you want to cache), or as the last element in the content array of a user message. It is not valid in assistant messages or in the middle of a content array followed by additional text blocks.

import {
  BedrockRuntimeClient,
  ConverseCommand,
  type ConverseCommandInput,
  type SystemContentBlock,
  type Message,
} from '@aws-sdk/client-bedrock-runtime';

const client = new BedrockRuntimeClient({ region: 'us-east-1' });

// System array with cachePoint after the stable system prompt
const system: SystemContentBlock[] = [
  {
    text: `You are an expert MCP server diagnostics assistant.
You help developers debug protocol errors, health check failures,
and authentication issues across all MCP transport types (SSE, stdio, HTTP).

When diagnosing issues, always:
1. Identify the specific MCP error code if mentioned (e.g. -32603, -32601)
2. Check for common SSE transport problems: connection drops, buffering, timeout
3. Suggest AliveMCP health check endpoint verification as first step
4. Provide actionable remediation steps in order of likelihood

Reference documentation:
${TOOL_DOCUMENTATION}  // ~3000 tokens of stable reference text
`,
  },
  // cachePoint after system text — everything above is checkpointed
  { cachePoint: { type: 'default' } } as unknown as SystemContentBlock,
];

// Per-turn user message — only the query is variable
const messages: Message[] = [
  {
    role: 'user',
    content: [{ text: userQuery }],  // just the user's question — NOT cached
  },
];

const input: ConverseCommandInput = {
  modelId: 'us.anthropic.claude-3-5-sonnet-20241022-v2:0',
  system,
  messages,
  inferenceConfig: { maxTokens: 2048 },
};

const response = await client.send(new ConverseCommand(input));

// Inspect cache efficiency
const usage = response.usage;
console.log({
  inputTokens:             usage?.inputTokens,
  cacheWriteTokenCount:    usage?.cacheWriteInputTokenCount,   // tokens written to cache (first request)
  cacheReadTokenCount:     usage?.cacheReadInputTokenCount,    // tokens read from cache (subsequent requests)
  outputTokens:            usage?.outputTokens,
});

On the first request, cacheWriteInputTokenCount will equal the token count of everything before the cachePoint. On subsequent requests within the 5-minute TTL window, cacheReadInputTokenCount equals that same count — and the cost for those tokens drops to 10% of the base input price. If cacheReadInputTokenCount stays zero after the first request, the cache is not being hit — see the troubleshooting section below.

Token thresholds per model

Bedrock enforces minimum token counts before a cachePoint can activate. Requests where the token count before the cachePoint is below the minimum are processed normally without caching — no error is raised, caching simply does not activate, and cacheWriteInputTokenCount will be 0.

For an MCP server using a 500-token system prompt and a 300-token tool schema block — 800 tokens total — caching will not activate on Claude 3.5 Sonnet (threshold: 2048). To reach the threshold you can either add more substantive content to the system prompt (documentation, examples, reference tables) or consolidate tool schemas into a larger single description block. Do not add filler text just to reach the threshold — it reduces model quality.

When content naturally falls below the minimum, omit the cachePoint entirely rather than leaving it in with no effect. A cachePoint that never activates adds a small serialization overhead with no benefit.

Caching tool schemas alongside the system prompt

MCP servers that act as Bedrock sub-agents pass tool schemas via toolConfig. Tool schemas are stable — they don't change between user requests — but they are passed as part of the toolConfig field, not as content blocks, so they cannot be directly targeted by a cachePoint in the standard Converse API flow.

The practical workaround is to embed a stringified summary of your tool schemas inside the system prompt text block, then place the cachePoint after that block. This way both the system instructions and the schema documentation are cached together.

// Build a stable system prompt that includes tool reference docs
function buildSystemPrompt(toolDefs: ToolSpec[]): string {
  const toolDocs = toolDefs.map(t =>
    `Tool: ${t.name}\nDescription: ${t.description}\n` +
    `Input schema: ${JSON.stringify(t.inputSchema.json, null, 2)}`
  ).join('\n\n---\n\n');

  return `${STATIC_SYSTEM_INSTRUCTIONS}\n\n## Available Tools\n\n${toolDocs}`;
}

// Build system array with cachePoint — generated once per server startup
const CACHED_SYSTEM: SystemContentBlock[] = [
  { text: buildSystemPrompt(MY_TOOL_SPECS) },
  { cachePoint: { type: 'default' } } as unknown as SystemContentBlock,
];

// Reuse CACHED_SYSTEM across all tool handler invocations
// The token content is identical across requests → cache hits
async function handleToolCall(userQuery: string) {
  return client.send(new ConverseCommand({
    modelId: 'us.anthropic.claude-3-5-sonnet-20241022-v2:0',
    system: CACHED_SYSTEM,
    messages: [{ role: 'user', content: [{ text: userQuery }] }],
    inferenceConfig: { maxTokens: 2048 },
  }));
}

The critical requirement for a cache hit is that the bytes before the cachePoint are byte-for-byte identical across requests. Even a single character difference — including whitespace, ordering, or version strings injected into the system prompt — produces a cache miss and a new cache write. For MCP servers: build the system prompt string once at startup and reuse the same object reference; do not re-interpolate per request.

Cache TTL and refresh behavior

Bedrock prompt cache entries expire 5 minutes after their last read. Each cache hit resets the 5-minute clock. For an MCP server receiving steady traffic (more than one call per 5 minutes to a given tool handler), the cache entry stays warm indefinitely at cache-read prices. For infrequently called tools, the first call in each 5-minute window pays the cache-write price; all subsequent calls within the window pay cache-read prices.

There is no API to extend TTL, manually evict entries, or query cache state. The 5-minute TTL is fixed. This has implications for:

• Low-traffic MCP servers: If your tool is called once every 10 minutes, every call will pay cache-write cost (125%). Caching provides no benefit unless call frequency exceeds one call per 5 minutes per cachePoint.
• Deployments across multiple availability zones: Cache entries are per-availability-zone. Requests routed to different AZs will each pay cache-write cost on their first request. High-traffic deployments should see AZ-level cache hits within seconds of the first request to each AZ.
• System prompt updates during deployment: Any change to the system prompt text invalidates the cache for all subsequent requests. Zero-downtime deployments with rolling restarts will see a brief period of increased cache-write costs as new instances build their cache entries.

Cost model and break-even analysis

Prompt caching pricing uses three tiers relative to the model's standard input token price:

Break-even calculation: if you send N requests per 5-minute TTL window, all sharing the same cache entry, the cost with caching is 1.25 × P_cached + (N-1) × 0.10 × P_cached + N × P_uncached where P_cached is the cost of the cached tokens and P_uncached is the cost of the non-cached tokens (user query, etc.). Without caching it is N × (P_cached + P_uncached). Break-even occurs at approximately N=1.4, meaning caching starts saving money after just 2 requests per 5-minute window that share the same cached prefix.

For an MCP server with a 3000-token system prompt at Claude 3.5 Sonnet pricing ($3/M input tokens): 3000 tokens × $0.000003/token = $0.009 per request without caching. With caching, first request = $0.01125 (125%), each subsequent = $0.0009 (10%). At 100 requests per 5-minute window: without caching = $0.90/window; with caching = $0.01125 + 99×$0.0009 = $0.10/window — an 89% reduction in system-prompt token costs.

Common placement mistakes and troubleshooting

Multi-turn conversations and cachePoint placement

For multi-turn agentic loops (where the MCP server calls Bedrock iteratively with accumulating message history), the cachePoint strategy changes. The system-level cachePoint stays constant across all turns. For the message history itself, you can add an additional cachePoint after the last assistant message to cache the accumulated conversation up to that point.

// Multi-turn loop with progressive caching
async function runAgentLoop(userQuery: string): Promise {
  const messages: Message[] = [];

  // Turn 1 — user query
  messages.push({ role: 'user', content: [{ text: userQuery }] });

  for (let turn = 0; turn < 10; turn++) {
    const response = await client.send(new ConverseCommand({
      modelId: 'us.anthropic.claude-3-5-sonnet-20241022-v2:0',
      system: CACHED_SYSTEM,   // stable system prompt with cachePoint
      messages,
      inferenceConfig: { maxTokens: 4096 },
    }));

    const outputMessage = response.output?.message!;
    messages.push(outputMessage);

    if (response.stopReason === 'end_turn') {
      return outputMessage.content
        ?.filter(b => 'text' in b)
        .map(b => (b as { text: string }).text)
        .join('') ?? '';
    }

    // Add cachePoint after the last assistant message to cache accumulated history
    // This is optional — only worth it if conversation history is > 2048 tokens
    if (turn > 0) {
      const lastAssistant = messages[messages.length - 1];
      if (lastAssistant.role === 'assistant') {
        // Add cachePoint as last block in next user message
        // to cache everything up to this point
        messages.push({
          role: 'user',
          content: [
            { text: await runToolAndGetResult(response) },
            { cachePoint: { type: 'default' } } as never,
          ],
        });
        continue;
      }
    }

    // Standard tool result injection without cachePoint
    messages.push({
      role: 'user',
      content: [{ text: await runToolAndGetResult(response) }],
    });
  }

  throw new Error('Agent loop exceeded max turns');
}

Use the message-level cachePoint conservatively: it only pays off when the accumulated history exceeds the minimum token threshold and when you expect multiple additional turns. For short conversations (2–3 turns), stick to the system-level cachePoint only.