S3 Event Notifications for MCP Servers

Notification destination comparison

S3 supports four notification destinations, each with different delivery semantics, latency, and filtering capabilities:

Choosing for MCP servers: if your MCP tool execution is slow (>30s) or bursty, SQS provides backpressure — your worker polls at its own rate without dropping events. If your tool is fast (<3s) and the bucket has low upload volume (<100 events/minute), Lambda is simpler. If the same S3 event must trigger multiple MCP tools (e.g., parse the document AND index the metadata AND notify the author), use SNS fan-out or EventBridge with multiple rules.

Configuring S3 → SQS notifications (CDK)

The most common pattern for MCP tool pipelines: uploads land in S3, an SQS queue buffers them, and your MCP worker polls the queue and executes the tool.

import * as cdk from 'aws-cdk-lib';
import * as s3 from 'aws-cdk-lib/aws-s3';
import * as sqs from 'aws-cdk-lib/aws-sqs';
import * as s3n from 'aws-cdk-lib/aws-s3-notifications';
import * as lambda from 'aws-cdk-lib/aws-lambda';
import * as lambdaEventSources from 'aws-cdk-lib/aws-lambda-event-sources';

// Dead-letter queue: capture events your worker fails to process
const dlq = new sqs.Queue(this, 'McpToolDlq', {
  retentionPeriod: cdk.Duration.days(14),
  encryption: sqs.QueueEncryption.SQS_MANAGED,
});

// Main processing queue
const processingQueue = new sqs.Queue(this, 'McpToolQueue', {
  visibilityTimeout: cdk.Duration.seconds(300),  // match your max tool execution time
  retentionPeriod: cdk.Duration.days(4),
  deadLetterQueue: { queue: dlq, maxReceiveCount: 3 },
  encryption: sqs.QueueEncryption.SQS_MANAGED,
});

// S3 bucket with event notifications
const uploadBucket = new s3.Bucket(this, 'McpUploadBucket', {
  encryption: s3.BucketEncryption.S3_MANAGED,
  blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL,
  versioned: true,
  lifecycleRules: [{
    id: 'expire-processed',
    prefix: 'processed/',
    expiration: cdk.Duration.days(30),
  }],
});

// Only fire on PDF uploads to the 'uploads/' prefix
uploadBucket.addEventNotification(
  s3.EventType.OBJECT_CREATED,
  new s3n.SqsDestination(processingQueue),
  { prefix: 'uploads/', suffix: '.pdf' },
);

// Also fire on CSV uploads (separate suffix filter — can't combine in one rule)
uploadBucket.addEventNotification(
  s3.EventType.OBJECT_CREATED,
  new s3n.SqsDestination(processingQueue),
  { prefix: 'uploads/', suffix: '.csv' },
);

Visibility timeout sizing: set visibility timeout to at least 6× the median tool execution time. If your MCP document-parsing tool takes 45 seconds, set visibility timeout to 300s (5 minutes). An event becomes visible again if not deleted within the timeout — you'll re-process it. The maxReceiveCount: 3 on the DLQ catches events that fail 3 times and moves them to the DLQ for inspection without losing them.

S3 notification limitation: each bucket can have at most 100 notification configurations. If you need more than 100 prefix/suffix combinations, use EventBridge (one rule per bucket, unlimited filter patterns per rule).

EventBridge: advanced filtering for MCP event routing

S3 → EventBridge delivers all S3 events to EventBridge first; you then write rules that filter and route to targets. The filter language is richer than native S3 notifications: you can match on object size, storage class, ETag, or any metadata field.

// Enable EventBridge notifications on the bucket (one-time setting)
const bucket = new s3.Bucket(this, 'McpDataBucket', {
  eventBridgeEnabled: true,  // CDK sets BucketNotificationConfiguration.EventBridgeConfiguration
  encryption: s3.BucketEncryption.S3_MANAGED,
  blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL,
});

import * as events from 'aws-cdk-lib/aws-events';
import * as targets from 'aws-cdk-lib/aws-events-targets';

// Rule 1: large files (>10MB) → slow processing queue
new events.Rule(this, 'LargeFileRule', {
  eventPattern: {
    source: ['aws.s3'],
    detailType: ['Object Created'],
    detail: {
      bucket: { name: [bucket.bucketName] },
      object: {
        key: [{ prefix: 'uploads/' }],
        size: [{ numeric: ['>=', 10_000_000] }],
      },
    },
  },
  targets: [new targets.SqsQueue(slowProcessingQueue)],
});

// Rule 2: small files (<10MB) → fast processing queue
new events.Rule(this, 'SmallFileRule', {
  eventPattern: {
    source: ['aws.s3'],
    detailType: ['Object Created'],
    detail: {
      bucket: { name: [bucket.bucketName] },
      object: {
        key: [{ prefix: 'uploads/' }],
        size: [{ numeric: ['<', 10_000_000] }],
      },
    },
  },
  targets: [new targets.SqsQueue(fastProcessingQueue)],
});

EventBridge rules fire in parallel — both rules above evaluate every event. There's no exclusive routing; if an event matches both rules, it's delivered to both targets. This is intentional for fan-out patterns: the same upload can simultaneously trigger a processing Lambda, a notification SNS topic, and an audit log Firehose.

EventBridge also enables cross-account delivery: a rule in account A can target an SQS queue or EventBridge bus in account B. This is essential for multi-tenant MCP architectures where tool execution runs in customer accounts but uploads land in a central data lake account.

Processing S3 events in a Node.js MCP worker

An MCP server that processes S3-triggered events needs to handle the SQS message envelope, extract the S3 event payload, and execute the tool. Key requirements: idempotency, error handling, and DLQ forwarding for permanent failures.

import { SQSClient, ReceiveMessageCommand, DeleteMessageCommand } from '@aws-sdk/client-sqs';
import { S3Client, GetObjectCommand } from '@aws-sdk/client-s3';
import { Readable } from 'stream';

const sqs = new SQSClient({ region: process.env.AWS_REGION });
const s3 = new S3Client({ region: process.env.AWS_REGION });

interface S3EventRecord {
  s3: {
    bucket: { name: string };
    object: { key: string; size: number; eTag: string };
  };
  eventName: string;
}

async function processS3Event(record: S3EventRecord): Promise {
  const bucket = record.s3.bucket.name;
  // URL-decode the object key (S3 encodes special chars and spaces as +)
  const key = decodeURIComponent(record.s3.object.key.replace(/\+/g, ' '));

  // Idempotency check: has this eTag already been processed?
  const eTag = record.s3.object.eTag.replace(/"/g, '');
  if (await alreadyProcessed(eTag)) {
    console.log(`Skipping duplicate event for eTag ${eTag}`);
    return;
  }

  const response = await s3.send(new GetObjectCommand({ Bucket: bucket, Key: key }));
  const body = await streamToBuffer(response.Body as Readable);

  // Execute MCP tool logic with the object content
  await runMcpTool(key, body, response.ContentType);

  // Mark as processed (idempotency record)
  await markProcessed(eTag);
}

async function pollQueue(): Promise {
  while (true) {
    const result = await sqs.send(new ReceiveMessageCommand({
      QueueUrl: process.env.SQS_QUEUE_URL!,
      MaxNumberOfMessages: 10,
      WaitTimeSeconds: 20,  // long polling — reduces API calls and cost
      VisibilityTimeout: 300,
    }));

    for (const msg of result.Messages ?? []) {
      const body = JSON.parse(msg.Body!);
      // SQS body wraps the S3 notification; SNS-wrapped events have an additional layer
      const s3Event = body.Records ? body : JSON.parse(body.Message);

      try {
        for (const record of s3Event.Records as S3EventRecord[]) {
          await processS3Event(record);
        }
        // Delete on success — removes from queue
        await sqs.send(new DeleteMessageCommand({
          QueueUrl: process.env.SQS_QUEUE_URL!,
          ReceiptHandle: msg.ReceiptHandle!,
        }));
      } catch (err) {
        // Don't delete — SQS will re-deliver after visibility timeout
        // After maxReceiveCount failures, message moves to DLQ automatically
        console.error('Tool execution failed:', err);
      }
    }
  }
}

The key decoding trap: S3 encodes object keys in event notifications using percent-encoding with + for spaces (not %20). Always apply decodeURIComponent(key.replace(/\+/g, ' ')) before using the key in a GetObject call. Missing this causes "NoSuchKey" errors for any object with a space, parenthesis, or special character in the name.

Test events: S3 sends a test notification when you first configure an event notification. These have an empty Records array. Your worker should handle this gracefully: if (!s3Event.Records?.length) return;.

Fan-out pattern: one upload → multiple MCP tools

A common architecture: when a file lands in S3, you want to run multiple independent tools in parallel — parse the document, generate embeddings, extract metadata, and notify the author. Use SNS fan-out for this:

// SNS topic as the fan-out hub
const fanoutTopic = new sns.Topic(this, 'S3FanoutTopic', {
  topicName: 'mcp-s3-fanout',
});

// S3 → SNS (single notification destination per bucket is the limit)
uploadBucket.addEventNotification(
  s3.EventType.OBJECT_CREATED,
  new s3n.SnsDestination(fanoutTopic),
  { prefix: 'uploads/' },
);

// SNS → Queue 1: document parsing worker
fanoutTopic.addSubscription(new snsSubscriptions.SqsSubscription(parseQueue, {
  filterPolicy: {
    // SNS message filter: only PDFs go to the parse queue
    // Note: SNS filter on S3 event attributes requires EventBridge; SNS filter works on
    // message attributes which S3 doesn't set — use Lambda to re-route based on content
  },
}));

// SNS → Queue 2: embedding generation worker
fanoutTopic.addSubscription(new snsSubscriptions.SqsSubscription(embedQueue));

// SNS → Queue 3: metadata indexer
fanoutTopic.addSubscription(new snsSubscriptions.SqsSubscription(indexQueue));

SNS filter limitation: SNS message attribute filters don't work with S3 event notifications because S3 doesn't set SNS message attributes on the notification messages. If you need content-based fan-out filtering (e.g., only PDFs go to the parse queue, only CSVs go to the data pipeline), use EventBridge instead of SNS — EventBridge rules support prefix/suffix filtering on the object key.

Event delivery guarantees and failure modes

S3 Event Notifications are at-least-once — in rare cases (S3 internal retry after a partial failure), the same event fires twice. Your MCP tool must be idempotent: processing the same S3 object twice should produce the same result as processing it once.

Known failure modes and mitigations:

The most common real-world failure is silent tool failures: the SQS message is received, your worker encounters an error, but the error handling swallows the exception and deletes the message anyway. The upload event is consumed but the tool never ran. Build your workers to only call DeleteMessage after confirmed successful tool execution — let visibility timeout + DLQ handle failures.

How AliveMCP monitors S3 event pipeline health

The hardest operational problem with S3 → SQS → MCP worker pipelines is that failures are silent: an upload lands in S3, the notification fires successfully, but the worker crashes mid-processing, and you only discover the failure when a user asks "why wasn't my document analyzed?" AliveMCP addresses this through three monitoring layers:

• SQS queue depth probe: AliveMCP polls the ApproximateNumberOfMessagesVisible and ApproximateAgeOfOldestMessage CloudWatch metrics for your processing queue. If messages accumulate beyond a threshold (queue depth >50 for >10 minutes), AliveMCP fires an alert — a worker is down or backed up.
• DLQ depth alert: any non-zero depth in your DLQ means events have failed 3 times and need investigation. AliveMCP monitors DLQ depth as a leading indicator of tool execution failures; even 1 DLQ message triggers an alert.
• End-to-end canary: on Team+ tier, AliveMCP uploads a canary file to your S3 bucket every 5 minutes and waits for a corresponding webhook confirmation that the file was processed. If the webhook doesn't arrive within the configured timeout, AliveMCP raises a "pipeline stalled" alert. This catches failures in S3 notification configuration, SQS delivery, and worker execution simultaneously.

Queue monitoring is available on the Author tier ($9/mo) via custom metric alerting. End-to-end canary testing requires Team tier ($49/mo) for the webhook ingestion infrastructure.