Guide · AWS SQS
SQS Visibility Timeout for MCP Servers — Lambda Relationship, ChangeMessageVisibility, Heartbeat Pattern
When a consumer receives an SQS message, that message becomes invisible to all other consumers for the duration of the visibility timeout. If the consumer deletes the message before the timeout expires, processing is complete. If the consumer crashes or the timeout expires before deletion, the message reappears in the queue and another consumer picks it up — this is SQS's at-least-once delivery guarantee. For MCP monitoring pipelines, misconfiguring visibility timeout is the most common cause of duplicate probe processing and ghost alerts: a probe result processed twice can flip a verdict from healthy back to a stale degraded state, or fire an alert for an endpoint that had already recovered. The three decisions that determine whether your pipeline handles visibility correctly are timeout duration relative to Lambda function timeout, whether to extend visibility for long-running consumers, and how Lambda's event source mapping manages the timeout on your behalf.
TL;DR
Set queue visibility timeout to at least 6× the Lambda function timeout (AWS recommendation). Lambda ESM automatically calls ChangeMessageVisibility on in-flight messages to extend the timeout as the function runs — you do not need to implement this yourself when using Lambda ESM. For long-running non-Lambda consumers (e.g., ECS probe workers), implement a heartbeat: extend visibility every timeout / 3 seconds to keep the message hidden while processing. Maximum single extension is 12 hours; the lifetime maximum total visibility for one receive attempt is also 12 hours.
How visibility timeout interacts with at-least-once delivery
The SQS message lifecycle has three key events:
- ReceiveMessage: the message is returned to the consumer and immediately hidden for
VisibilityTimeoutseconds. - DeleteMessage: the consumer calls this after successfully processing. The message is permanently removed.
- Timeout expiry: if DeleteMessage is never called, the message becomes visible again. Its
ApproximateReceiveCountincrements.
When ApproximateReceiveCount exceeds the dead-letter queue's maxReceiveCount, the message is sent to the DLQ instead of being re-enqueued. This is the only automatic failsafe against a poison-pill message — one that always causes the consumer to crash — looping forever in the queue.
| Scenario | Message fate | Risk for MCP pipelines |
|---|---|---|
| Consumer processes and deletes within timeout | Permanently removed | None |
| Consumer crashes before delete | Reappears in queue after timeout | Message reprocessed — can cause duplicate alert |
| Consumer takes longer than timeout | Reappears while original consumer still processing | Two consumers process same probe result simultaneously — race condition on verdict store |
| Consumer network partition (cannot delete) | Reappears after timeout | Same as crash — duplicate processing |
Visibility timeout and Lambda function timeout
The critical constraint: if the Lambda function timeout fires while a message is still being processed, Lambda does not automatically delete or extend the message — the function execution terminates, and the message becomes visible again.
AWS's documented recommendation is to set the queue visibility timeout to at least 6× the Lambda function timeout. The rationale: if Lambda retries the invocation (rare but possible on internal errors), the second invocation has time to complete without racing the first one's visibility window.
| Lambda function timeout | Recommended queue visibility timeout |
|---|---|
| 5 seconds (fast probe health check) | 30 seconds |
| 15 seconds (standard probe) | 90 seconds (round up to 2 minutes) |
| 30 seconds | 3 minutes |
| 60 seconds | 6 minutes |
| 5 minutes (max for most Lambda workloads) | 30 minutes |
| 15 minutes (Lambda max) | 12 hours (use heartbeat instead) |
import { Queue, Duration } from 'aws-cdk-lib/aws-sqs';
import { Function, Runtime } from 'aws-cdk-lib/aws-lambda';
import { SqsEventSource } from 'aws-cdk-lib/aws-lambda-event-sources';
const lambdaTimeout = Duration.seconds(30);
const probeQueue = new Queue(this, 'ProbeQueue', {
visibilityTimeout: Duration.seconds(lambdaTimeout.toSeconds() * 6), // 180s
deadLetterQueue: {
queue: dlq,
maxReceiveCount: 3,
},
});
const probeFunction = new Function(this, 'ProbeFunction', {
timeout: lambdaTimeout,
// ...
});
probeFunction.addEventSource(new SqsEventSource(probeQueue, {
batchSize: 1,
// Lambda ESM manages visibility extension automatically
}));
Lambda ESM automatic visibility extension
When Lambda processes SQS messages via an Event Source Mapping (ESM), it automatically calls ChangeMessageVisibility on each in-flight message at roughly two-thirds of the way through the function timeout. This extends the visibility window, preventing the message from reappearing while the function is still running.
Specifically, if the function timeout is T seconds, Lambda ESM extends visibility at approximately 2T/3 seconds into the invocation. The extension sets a new visibility timeout equal to the queue's configured VisibilityTimeout — not the remaining function time. This can create an unexpectedly long re-hide window if your queue's visibility timeout is large.
What Lambda ESM does not do:
- It does not extend visibility for messages in a batch where individual messages take varying amounts of time — the extension applies to all messages in the batch at once.
- It does not call
ChangeMessageVisibilitymore than once per invocation. If your function runs longer than 2× the function timeout (possible with certain async patterns), the message may reappear. - It does not extend visibility when the function is in the process of terminating due to a timeout error.
ChangeMessageVisibility: extending manually for non-Lambda consumers
For ECS-based probe workers or Node.js/Go processes that poll SQS directly, you must extend visibility manually. The ChangeMessageVisibility call updates the timeout from the current moment — it does not add to the remaining time, it resets it.
import { SQSClient, ChangeMessageVisibilityCommand, DeleteMessageCommand } from '@aws-sdk/client-sqs';
const sqs = new SQSClient({});
const VISIBILITY_TIMEOUT_SEC = 60; // queue setting
const HEARTBEAT_INTERVAL_SEC = 20; // extend every 1/3 of the timeout
async function processWithHeartbeat(
queueUrl: string,
receiptHandle: string,
work: () => Promise
): Promise {
let heartbeatTimer: NodeJS.Timeout | undefined;
const heartbeat = async () => {
try {
await sqs.send(new ChangeMessageVisibilityCommand({
QueueUrl: queueUrl,
ReceiptHandle: receiptHandle,
VisibilityTimeout: VISIBILITY_TIMEOUT_SEC, // reset to full timeout
}));
} catch (err) {
// If the message is already deleted, ReceiptHandleIsInvalid — stop heartbeat
clearInterval(heartbeatTimer!);
}
};
heartbeatTimer = setInterval(heartbeat, HEARTBEAT_INTERVAL_SEC * 1000);
try {
await work();
await sqs.send(new DeleteMessageCommand({ QueueUrl: queueUrl, ReceiptHandle: receiptHandle }));
} finally {
clearInterval(heartbeatTimer);
}
}
The heartbeat interval should be visibilityTimeout / 3: frequent enough to avoid expiry, infrequent enough to avoid unnecessary API calls. SQS charges per API call, so a 5-second heartbeat on a high-volume queue is expensive. For a 60-second visibility timeout, a 20-second heartbeat is appropriate.
Maximum visibility timeout and 12-hour limit
The maximum visibility timeout you can set on a queue is 12 hours. The maximum value for a single ChangeMessageVisibility call is also 12 hours. However, the total visibility duration across all extensions for one receive attempt is capped at 12 hours from the initial receive. After 12 hours, the message reappears regardless of extensions.
For MCP probe pipelines with processing times in the seconds-to-minutes range, the 12-hour cap is never relevant. It matters only for batch-archival or report-generation consumers that may run for hours.
Common failure modes
| Symptom | Cause | Fix |
|---|---|---|
| Same probe result processed twice, duplicate alert fired | Visibility timeout shorter than function execution time — message reappears while first consumer is still processing | Increase visibility timeout to 6× Lambda timeout; or reduce function timeout; or reduce batch size to 1 to minimize individual message processing time |
| Message never deleted but never reaches DLQ | Consumer silently swallowing delete error; or consumer crashed after extending visibility but before deleting | Always call DeleteMessage in a finally block; log and alarm on delete errors |
Heartbeat fails with ReceiptHandleIsInvalid | Message was deleted successfully by a concurrent consumer after the visibility window expired | Catch this error specifically and stop the heartbeat — it means the message was processed by another consumer |
| Lambda ESM message reappears despite function completing within timeout | Queue visibility timeout shorter than Lambda function timeout — ESM extension fires but cannot set visibility below function timeout | Queue visibility timeout must be ≥ Lambda function timeout; the 6× rule is a safety margin, not a hard minimum |
| Probe results arriving at DLQ after only one or two receive attempts | maxReceiveCount too low combined with transient Lambda cold-start timeouts | Increase maxReceiveCount to 3–5; Lambda cold starts are not failures, they are retries |