MCP Server SQS FIFO Queue — MessageGroupId, deduplication, ordering guarantees

When to use FIFO vs standard queues for MCP tool calls

Most MCP tool calls are independent: a web_search tool, a read_file tool, and a send_email tool called in the same agent turn can execute concurrently without coordination. Standard queues work well here — higher throughput, no ordering constraints, simpler configuration.

FIFO queues are appropriate when tool calls within a session have a dependency chain and must execute sequentially. Examples:

• A create_document tool followed immediately by append_section — the second call must execute after the first has completed, not in parallel.
• A database tool that runs BEGIN TRANSACTION, multiple INSERT calls, then COMMIT — the SQL statements must arrive at the worker in order.
• A multi-step workflow tool that advances a state machine — calling step 3 before step 2 puts the machine in an invalid state.

In all three cases, use FIFO with MessageGroupId = sessionId: all tool calls for a session go to the same group, and SQS guarantees they are delivered to consumers in the order they were sent.

MessageGroupId: choosing the right granularity

MessageGroupId is required for every message sent to a FIFO queue. SQS delivers messages within a group in strict FIFO order; across groups, delivery order is not guaranteed. One consumer can hold one message group at a time — if a consumer is processing a message from group session-abc, no other consumer receives another message from session-abc until the first message is deleted or its visibility timeout expires.

This makes the choice of granularity a throughput-vs-ordering trade-off:

import {
  SQSClient,
  SendMessageCommand,
  type SendMessageCommandInput,
} from '@aws-sdk/client-sqs';

const sqs = new SQSClient({ region: process.env.AWS_REGION ?? 'us-east-1' });
const FIFO_QUEUE_URL = process.env.FIFO_QUEUE_URL!; // must end in .fifo

interface OrderedToolCall {
  toolCallId: string;
  toolName: string;
  toolArgs: unknown;
  sessionId: string;
  sequenceNumber: number; // monotonically increasing within session
}

async function enqueueOrderedToolCall(call: OrderedToolCall): Promise {
  const input: SendMessageCommandInput = {
    QueueUrl: FIFO_QUEUE_URL,
    MessageBody: JSON.stringify(call),
    MessageGroupId: call.sessionId,           // per-session FIFO ordering
    MessageDeduplicationId: call.toolCallId,  // exact-once delivery by ID
  };

  await sqs.send(new SendMessageCommand(input));
}

// Receiving ordered tool calls — within a session group, messages arrive in send order
import { ReceiveMessageCommand, DeleteMessageCommand } from '@aws-sdk/client-sqs';

async function pollFifoQueue(): Promise {
  while (true) {
    const response = await sqs.send(new ReceiveMessageCommand({
      QueueUrl: FIFO_QUEUE_URL,
      MaxNumberOfMessages: 10,
      WaitTimeSeconds: 20,
      MessageAttributeNames: ['All'],
      AttributeNames: ['MessageGroupId', 'SequenceNumber', 'ApproximateReceiveCount'],
    }));

    if (!response.Messages?.length) continue;

    // Messages from the same group arrive in FIFO order; process sequentially within group
    for (const message of response.Messages) {
      await processOrderedToolCall(message);
      await sqs.send(new DeleteMessageCommand({
        QueueUrl: FIFO_QUEUE_URL,
        ReceiptHandle: message.ReceiptHandle!,
      }));
    }
  }
}

Deduplication: ContentBasedDeduplication vs MessageDeduplicationId

FIFO queues require a deduplication ID for each message to provide exactly-once delivery. SQS uses a 5-minute deduplication window: if two messages with the same deduplication ID are sent within 5 minutes, the second is silently discarded.

ContentBasedDeduplication (enabled at queue creation with ContentBasedDeduplication: true) derives the deduplication ID automatically from a SHA-256 hash of the message body. This is convenient but breaks when message bodies are not deterministic — for example, if you include a sentAt timestamp, request ID, or any other field that changes between retries. Two semantically identical tool calls with different timestamps will NOT be deduplicated because their bodies differ.

MessageDeduplicationId (provided by the sender on each SendMessage call) is explicit. Use the toolCallId UUID that the MCP server generates when it receives the tools/call RPC — this ID is stable across retries and exactly identifies the logical tool call invocation. This is the correct approach for MCP tool queuing.

// CORRECT: explicit deduplication ID using the stable toolCallId
await sqs.send(new SendMessageCommand({
  QueueUrl: FIFO_QUEUE_URL,
  MessageBody: JSON.stringify({
    toolCallId: 'tc_01j8x9y2z3a4b5c6d7e8f9',
    toolName: 'create_document',
    toolArgs: { title: 'Q3 Report', content: '...' },
    sessionId: 'sess_abc123',
    sentAt: Date.now(), // non-deterministic, but doesn't affect dedup since ID is explicit
  }),
  MessageGroupId: 'sess_abc123',
  MessageDeduplicationId: 'tc_01j8x9y2z3a4b5c6d7e8f9', // stable toolCallId
}));

// WRONG: ContentBasedDeduplication with non-deterministic body
// sentAt changes between retries → different SHA-256 → no deduplication
// A retry sends an identical tool call that gets executed twice
await sqs.send(new SendMessageCommand({
  QueueUrl: FIFO_QUEUE_URL,
  MessageBody: JSON.stringify({
    toolName: 'create_document',
    toolArgs: { title: 'Q3 Report' },
    sentAt: Date.now(), // changes on every retry
  }),
  MessageGroupId: 'sess_abc123',
  // No MessageDeduplicationId — relies on ContentBasedDeduplication
  // Two retries of the same call will NOT be deduplicated
}));

// FIFO queue with ContentBasedDeduplication disabled (explicit IDs required):
// aws sqs create-queue \
//   --queue-name tool-calls.fifo \
//   --attributes FifoQueue=true,ContentBasedDeduplication=false,VisibilityTimeout=360

Throughput limits and high-throughput FIFO mode

Standard FIFO queues support 300 messages per second per queue (3,000 messages/s when using batched SendMessageBatch with 10 messages per call). This limit applies to the entire queue, not per message group. At 3,000 messages/s with batching, a single FIFO queue can handle roughly 260 million tool calls per day — sufficient for most MCP deployments.

When you need higher throughput, enable high-throughput FIFO mode by setting two queue attributes:

• DeduplicationScope=messageGroup — deduplication is applied per message group instead of per queue, enabling parallel processing across groups.
• FifoThroughputLimit=perMessageGroupId — the 300 msg/s limit applies per message group, not per queue.

With high-throughput FIFO enabled, a queue with 1,000 active message groups achieves up to 300,000 messages/s. The ordering and exactly-once guarantees remain in effect within each message group.

// Enable high-throughput FIFO at creation:
// aws sqs create-queue \
//   --queue-name tool-calls-ht.fifo \
//   --attributes 'FifoQueue=true,ContentBasedDeduplication=false,\
//     DeduplicationScope=messageGroup,\
//     FifoThroughputLimit=perMessageGroupId,\
//     VisibilityTimeout=360'

// Or update an existing queue:
import { SetQueueAttributesCommand } from '@aws-sdk/client-sqs';

await sqs.send(new SetQueueAttributesCommand({
  QueueUrl: FIFO_QUEUE_URL,
  Attributes: {
    DeduplicationScope: 'messageGroup',
    FifoThroughputLimit: 'perMessageGroupId',
  },
}));

// Note: high-throughput FIFO requires ContentBasedDeduplication=false.
// You cannot enable high-throughput mode and ContentBasedDeduplication simultaneously.
// This is another reason to always use explicit MessageDeduplicationId.

One important subtlety: when DeduplicationScope=messageGroup, the 5-minute deduplication window applies per message group, not per queue. If you accidentally send the same MessageDeduplicationId in different message groups, both messages are delivered — the deduplication check is group-scoped, not queue-scoped. Keep MessageDeduplicationId globally unique (use UUIDs, not sequential integers or tool-name-based strings).

FIFO and Lambda event source mappings

When a Lambda function consumes a FIFO queue via an event source mapping, Lambda delivers batches from a single message group at a time. Within a batch, all messages are from the same group and are in FIFO order. Lambda processes up to batchSize messages in sequence, then deletes them. If a message fails (the function throws), Lambda does NOT move to the next message in the group — it retries the entire batch or moves the failing batch to a DLQ. This means a single bad message blocks all subsequent messages in that group until it exhausts maxReceiveCount and moves to the DLQ.

Set FunctionResponseTypes: ['ReportBatchItemFailures'] and return a batchItemFailures response to report exactly which messages failed, so SQS can retry individual messages instead of the entire batch. This is especially important for FIFO where a batch failure blocks the entire group.

// Lambda handler for FIFO queue — report partial batch failures
interface SQSRecord {
  messageId: string;
  receiptHandle: string;
  body: string;
  attributes: { MessageGroupId: string; SequenceNumber: string; ApproximateReceiveCount: string };
}

interface SQSEvent {
  Records: SQSRecord[];
}

interface BatchItemFailure {
  itemIdentifier: string;
}

export async function handler(event: SQSEvent): Promise<{ batchItemFailures: BatchItemFailure[] }> {
  const failures: BatchItemFailure[] = [];

  for (const record of event.Records) {
    try {
      const payload = JSON.parse(record.body) as OrderedToolCall;
      await processOrderedToolCall(payload);
    } catch (err) {
      console.error({ messageId: record.messageId, error: String(err) });
      failures.push({ itemIdentifier: record.messageId });
      // For FIFO: failure here stops processing subsequent messages in this group
      // Return immediately so later messages in the group are not attempted
      break;
    }
  }

  return { batchItemFailures: failures };
}

Failure modes reference