MCP Server SQS Tool Queue — visibility timeout, long polling, dead-letter queue

Why SQS for MCP tool calls

MCP tool handlers in a standard server run synchronously inside the JSON-RPC request-response cycle. The MCP client sends a tools/call request and blocks waiting for the response. For tools that complete in under a second this is fine. For tools that invoke external APIs (web search, database queries, code execution, image generation) the synchronous model creates three problems:

Protocol timeout: MCP clients impose a timeout on tool calls — typically 30–120 seconds depending on the client and configuration. A tool that calls an LLM API for chain-of-thought reasoning or runs a multi-step database migration can easily exceed this. The client receives a timeout error; the tool worker continues running; the result is silently discarded.

Concurrency pressure: An agent that fans out 10 parallel tool calls saturates the MCP server's connection pool or thread pool. If the server is single-process Node.js, all 10 tool calls share the event loop — a single blocking operation stalls all of them.

Retry amplification: When a downstream API throttles a tool call, the MCP client retries the entire tools/call request. Without an intermediate queue the MCP server cannot absorb retries — it just proxies the retry straight to the throttled API, making throttling worse.

An SQS queue between the MCP server and the tool workers solves all three: the server accepts the tools/call immediately and returns a pending token; workers process at their own pace; retries are managed at the queue layer with backoff.

import {
  SQSClient,
  SendMessageCommand,
  ReceiveMessageCommand,
  DeleteMessageCommand,
  ChangeMessageVisibilityCommand,
  type SendMessageCommandInput,
  type Message,
} from '@aws-sdk/client-sqs';

const sqs = new SQSClient({ region: process.env.AWS_REGION ?? 'us-east-1' });
const TOOL_QUEUE_URL = process.env.TOOL_QUEUE_URL!; // required env var

// Minimum IAM permissions for producers (MCP server):
// sqs:SendMessage on the queue ARN
// Minimum IAM permissions for consumers (tool workers):
// sqs:ReceiveMessage, sqs:DeleteMessage, sqs:ChangeMessageVisibility on the queue ARN

Visibility timeout: the most common misconfiguration

When a consumer calls ReceiveMessage, SQS makes the message invisible to all other consumers for VisibilityTimeout seconds. If the consumer does not delete the message before the timeout expires, SQS makes the message visible again — another consumer picks it up. This is the SQS "at-least-once" delivery guarantee. For tool calls the consequence is duplicate execution: a slow tool worker that hasn't finished when the timeout expires will see the same tool call dispatched to a second worker simultaneously.

The default VisibilityTimeout on a new SQS queue is 30 seconds. For a tool that calls an external LLM API (which can take 10–45 seconds), this means duplicate execution is almost certain on any call longer than 30 seconds.

Calculate visibility timeout as: max_expected_tool_duration × 3. The factor-of-three buffer covers: the worker's processing time, any downstream retry backoff the worker does internally, and the time to write the result back to the reply store. If your slowest tool can take 2 minutes, set VisibilityTimeout to at least 360 seconds (6 minutes). The maximum is 12 hours (43,200 seconds).

// Queue configuration — set at queue creation or update
// aws sqs set-queue-attributes --queue-url $URL \
//   --attributes VisibilityTimeout=360,MessageRetentionPeriod=86400

// Within a consumer: extend visibility before it expires if work is taking longer
async function processWithExtension(message: Message): Promise {
  const receiptHandle = message.ReceiptHandle!;
  const queueUrl = TOOL_QUEUE_URL;

  // Extend every 2 minutes while work is running
  const extendInterval = setInterval(async () => {
    await sqs.send(new ChangeMessageVisibilityCommand({
      QueueUrl: queueUrl,
      ReceiptHandle: receiptHandle,
      VisibilityTimeout: 360, // reset to full 6-minute window
    }));
  }, 120_000); // 2 minutes

  try {
    await executeTool(message);
    await sqs.send(new DeleteMessageCommand({
      QueueUrl: queueUrl,
      ReceiptHandle: receiptHandle,
    }));
  } finally {
    clearInterval(extendInterval);
  }
}

// WRONG — extends visibility, but after the timeout already expired.
// Once VisibilityTimeout lapses, ReceiptHandle is invalid:
// ChangeMessageVisibility returns InvalidParameterValue.
// Always extend BEFORE expiry, not after.
async function wrongExtend(receiptHandle: string): Promise {
  await new Promise(r => setTimeout(r, 400_000)); // wait 400s > 360s timeout
  // This call will throw InvalidParameterValue:
  await sqs.send(new ChangeMessageVisibilityCommand({
    QueueUrl: TOOL_QUEUE_URL,
    ReceiptHandle: receiptHandle,
    VisibilityTimeout: 360,
  }));
}

Note: if a Lambda function is consuming the queue via an event source mapping, Lambda manages visibility timeout extension automatically during function execution. The ChangeMessageVisibilityCommand pattern above is for polling consumers (non-Lambda workers) where you control the poll loop.

Long polling: always set WaitTimeSeconds to 20

Short polling (ReceiveMessage without WaitTimeSeconds, or with WaitTimeSeconds: 0) returns immediately whether or not messages are available. If the queue is empty, SQS queries a subset of its distributed storage backends and returns an empty response — consuming one API call and one request unit. A consumer that polls in a tight loop burns roughly 144,000 API calls per day doing nothing but confirming the queue is empty.

Long polling sets WaitTimeSeconds to a value between 1 and 20. SQS holds the connection open and returns as soon as a message arrives, or after WaitTimeSeconds seconds if no message arrives. With WaitTimeSeconds: 20, an idle consumer makes at most 4,320 API calls per day instead of 144,000 — a 33× reduction in SQS costs for quiet queues.

Long polling also reduces false empty responses. Short polling queries a random subset of SQS's distributed storage nodes; a message newly enqueued to nodes not in that subset returns an empty response even though messages exist. Long polling queries all storage nodes, eliminating the false-empty window.

// Consumer poll loop with long polling and batch receipt
async function pollToolQueue(): Promise {
  while (true) {
    const response = await sqs.send(new ReceiveMessageCommand({
      QueueUrl: TOOL_QUEUE_URL,
      MaxNumberOfMessages: 10,          // up to 10 per call (SQS max)
      WaitTimeSeconds: 20,              // long poll — ALWAYS set this
      MessageAttributeNames: ['All'],   // required — default returns no attributes
      AttributeNames: ['ApproximateReceiveCount', 'SentTimestamp'],
    }));

    if (!response.Messages || response.Messages.length === 0) continue;

    await Promise.all(response.Messages.map(processWithExtension));
  }
}

// Sending a tool call to the queue from the MCP server
interface ToolCallPayload {
  toolCallId: string;   // idempotency key
  toolName: string;
  toolArgs: unknown;
  sessionId: string;
  replyKey: string;     // where to write the result (e.g., Redis key or DynamoDB PK)
}

async function enqueueToolCall(payload: ToolCallPayload): Promise {
  const input: SendMessageCommandInput = {
    QueueUrl: TOOL_QUEUE_URL,
    MessageBody: JSON.stringify(payload),
    MessageAttributes: {
      toolName: { DataType: 'String', StringValue: payload.toolName },
      sessionId: { DataType: 'String', StringValue: payload.sessionId },
    },
    // MessageDeduplicationId only for FIFO queues.
    // For standard queues, use toolCallId in the body for app-level dedup.
  };

  const { MessageId } = await sqs.send(new SendMessageCommand(input));
  return MessageId!;
}

Dead-letter queue: configure before you need it

A dead-letter queue (DLQ) is a second SQS queue that receives messages after they have been delivered more than maxReceiveCount times without being deleted. Without a DLQ, a message that always fails processing is re-queued indefinitely. It circulates through your queue until its MessageRetentionPeriod (default 4 days) expires, consuming receive-count quota, triggering worker restarts, and masking the root cause because there is no way to inspect messages in flight.

A DLQ that catches poison pills lets you inspect the messages that failed, understand the failure pattern (bad JSON, missing required field, downstream API always returning 500), and replay them via StartMessageMoveTask after fixing the underlying issue.

// CloudFormation / Terraform: set redrive policy on the source queue
// SourceQueue.RedrivePolicy:
//   deadLetterTargetArn: !GetAtt ToolDLQ.Arn
//   maxReceiveCount: 3

// In the CDK:
import { Queue } from 'aws-cdk-lib/aws-sqs';
import { Duration } from 'aws-cdk-lib';

const dlq = new Queue(this, 'ToolCallDLQ', {
  retentionPeriod: Duration.days(14),
  queueName: 'tool-calls-dlq',
});

const toolQueue = new Queue(this, 'ToolCallQueue', {
  visibilityTimeout: Duration.seconds(360),
  retentionPeriod: Duration.days(4),
  receiveMessageWaitTime: Duration.seconds(20), // queue-level default long poll
  deadLetterQueue: {
    queue: dlq,
    maxReceiveCount: 3, // after 3 failed deliveries, move to DLQ
  },
});

// DLQ alarm: alert when any message lands in the DLQ
import { Alarm, Metric } from 'aws-cdk-lib/aws-cloudwatch';
new Alarm(this, 'DLQAlarm', {
  metric: dlq.metricNumberOfMessagesSent(),
  threshold: 1,
  evaluationPeriods: 1,
  alarmDescription: 'Tool call DLQ received a message — investigate immediately',
});

The maxReceiveCount value controls how many attempts a worker gets before a message is moved to the DLQ. The ApproximateReceiveCount message attribute (available when you pass AttributeNames: ['ApproximateReceiveCount'] in ReceiveMessage) tells a consumer how many times the message has already been delivered. Use it to detect near-DLQ messages and log extra diagnostics before the final attempt:

async function processWithDiagnostics(message: Message): Promise {
  const receiveCount = parseInt(message.Attributes?.ApproximateReceiveCount ?? '1', 10);

  if (receiveCount >= 2) {
    // Near DLQ threshold — emit structured warning with full payload
    console.warn({
      event: 'tool_call_near_dlq',
      receiveCount,
      messageId: message.MessageId,
      body: message.Body,
    });
  }

  // ... process the tool call ...
  await sqs.send(new DeleteMessageCommand({
    QueueUrl: TOOL_QUEUE_URL,
    ReceiptHandle: message.ReceiptHandle!,
  }));
}

Message retention and idempotency

Message retention period (default 4 days, max 14 days) is how long SQS keeps a message if it is never received and deleted. For tool calls this is usually not an issue — tool call results become meaningless after the session ends. Set it to match your session timeout plus some buffer: if MCP sessions live at most 1 hour, a 1-day retention period is generous. Longer retention inflates storage costs for queues with high message volume.

Idempotency is required for any queue-backed tool call worker because SQS standard queues deliver messages at-least-once. A second delivery of the same message should not cause the tool to execute twice — especially for write operations, billing events, or any tool that has external side effects.

The idempotency pattern: include a toolCallId (a UUID generated by the MCP server before enqueue) in every message body. Before executing the tool, the worker checks a short-lived store (DynamoDB with TTL, Redis with EX, or a local in-memory map for single-process workers) for that ID. If found, skip execution and return the cached result. If not found, execute, store the result keyed by toolCallId, and delete the SQS message.

import { DynamoDBClient, GetItemCommand, PutItemCommand } from '@aws-sdk/client-dynamodb';
import { marshall, unmarshall } from '@aws-sdk/util-dynamodb';

const dynamo = new DynamoDBClient({});
const IDEMPOTENCY_TABLE = process.env.IDEMPOTENCY_TABLE!;
const TTL_SECONDS = 3600; // 1 hour

async function idempotentToolCall(
  toolCallId: string,
  execute: () => Promise,
): Promise {
  // Check for cached result
  const existing = await dynamo.send(new GetItemCommand({
    TableName: IDEMPOTENCY_TABLE,
    Key: marshall({ toolCallId }),
    ConsistentRead: true, // must be consistent to catch concurrent duplicates
  }));

  if (existing.Item) {
    const item = unmarshall(existing.Item);
    return JSON.parse(item.result);
  }

  const result = await execute();

  // Store result — condition prevents overwrite if concurrent duplicate raced us
  await dynamo.send(new PutItemCommand({
    TableName: IDEMPOTENCY_TABLE,
    Item: marshall({
      toolCallId,
      result: JSON.stringify(result),
      ttl: Math.floor(Date.now() / 1000) + TTL_SECONDS,
    }),
    ConditionExpression: 'attribute_not_exists(toolCallId)',
  }).catch(() => {
    // ConditionalCheckFailedException means duplicate won the race — that's fine
  }));

  return result;
}

Failure modes reference