AWS Networking & Security · 2026-08-27 · AWS Networking & Security arc
AWS Networking and Security for MCP Servers: VPC, ALB, CloudFront, Cognito, and WAF — Five Layers of Production Infrastructure
Running an MCP server in production on AWS means layering five distinct networking and security concerns correctly. Each layer has its own failure modes that are nearly invisible until you're under load or being attacked. This post synthesizes the full stack: the VPC foundation that routes and isolates traffic, the ALB configuration that exposes your MCP endpoint correctly, the CloudFront split that caches static content without buffering SSE streams, the Cognito authentication layer that enforces per-tool OAuth scopes, and the WAF hardening that blocks abuse without false-positiving on legitimate MCP payloads. Getting any one of these wrong creates a failure that the others cannot compensate for.
Pattern 1 — VPC foundation: private subnets, security group chain, NAT vs endpoints
The VPC is the security and cost foundation for everything that runs on ECS Fargate. Two common first-deployment mistakes are placing ECS tasks in public subnets (which exposes the task's ENI directly to the internet — the ALB is the only entry point that should be public-facing) and forgetting that Fargate tasks in private subnets have no route to the public ECR endpoint without a NAT gateway or VPC endpoints.
Subnet layout: two tiers, one direction of traffic
The standard layout uses two subnet tiers across at least two Availability Zones. Public subnets hold the ALB and NAT gateways. Private subnets hold ECS tasks (and optionally RDS, ElastiCache, or other backends). Traffic flows inbound from the internet through the ALB, and outbound from tasks through the NAT gateway or VPC endpoints.
// CDK: VPC with public and private subnets across 2 AZs
import * as ec2 from "aws-cdk-lib/aws-ec2";
const vpc = new ec2.Vpc(this, "McpVpc", {
maxAzs: 2,
natGateways: 1, // single NAT gateway: lower cost, reduced HA
// natGateways: 2, // one per AZ: full HA, $0.045/hr per gateway extra
subnetConfiguration: [
{
name: "Public",
subnetType: ec2.SubnetType.PUBLIC,
cidrMask: 24,
},
{
name: "Private",
subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS,
cidrMask: 24,
},
],
enableDnsHostnames: true, // REQUIRED for Interface VPC endpoint private DNS
enableDnsSupport: true, // REQUIRED for VPC DNS resolver
});
// ECS service: private subnets only, no public IP
const service = new ecs.FargateService(this, "McpService", {
cluster,
taskDefinition,
vpcSubnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS },
assignPublicIp: false,
});
The two DNS options — enableDnsHostnames and enableDnsSupport — are both required for Interface VPC endpoints to work. Without enableDnsHostnames, the endpoint's private DNS override doesn't apply, and SDK calls from your tasks still route to the public endpoint (bypassing the VPC endpoint and going through the NAT gateway, incurring data-processing charges).
Security group chain: ALB → task → endpoints
The security group design follows a chain: the ALB accepts public HTTPS traffic; the task security group accepts traffic only from the ALB security group; the endpoint security group accepts HTTPS from the task security group. Nothing else reaches the tasks.
| Security group | Inbound rule | Outbound rule | Purpose |
|---|---|---|---|
albSg | TCP 443 from 0.0.0.0/0 | TCP 3000 to taskSg | Public HTTPS entry point |
albSg | TCP 80 from 0.0.0.0/0 (redirect only) | — | HTTP → HTTPS redirect listener |
taskSg | TCP 3000 from albSg | TCP 443 to 0.0.0.0/0 | MCP container: receives traffic only from ALB |
endpointSg | TCP 443 from taskSg | — | VPC Interface endpoints: accept HTTPS from tasks only |
// CDK: security group chain
const albSg = new ec2.SecurityGroup(this, "AlbSg", {
vpc,
allowAllOutbound: false,
});
albSg.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(443), "HTTPS from internet");
albSg.addIngressRule(ec2.Peer.anyIpv4(), ec2.Port.tcp(80), "HTTP redirect");
const taskSg = new ec2.SecurityGroup(this, "TaskSg", {
vpc,
allowAllOutbound: true, // tasks need outbound for ECR pull, SSM, Secrets Manager
});
taskSg.addIngressRule(albSg, ec2.Port.tcp(3000), "ALB to MCP container");
albSg.addEgressRule(taskSg, ec2.Port.tcp(3000), "ALB to MCP task");
const endpointSg = new ec2.SecurityGroup(this, "EndpointSg", {
vpc,
allowAllOutbound: false,
});
endpointSg.addIngressRule(taskSg, ec2.Port.tcp(443), "tasks to VPC endpoints");
NAT gateway vs. VPC endpoints: cost model
Every AWS API call from a task in a private subnet routes through the NAT gateway if no VPC endpoint exists for that service. NAT gateway charges $0.045/GB of data processed. ECR image pulls, SSM parameter reads, Secrets Manager secret fetches, and CloudWatch log writes all route through NAT without endpoints.
| Endpoint | Type | Hourly cost | Why needed |
|---|---|---|---|
| S3 | Gateway | Free | ECR image layers are stored in S3; required for all image pulls |
| ECR API | Interface | $0.01/hr (~$7.20/month) | GetAuthorizationToken and image metadata calls |
| ECR Docker | Interface | $0.01/hr (~$7.20/month) | Docker layer pulls from ECR |
| SSM | Interface | $0.01/hr (~$7.20/month) | GetParametersByPath at startup |
| Secrets Manager | Interface | $0.01/hr (~$7.20/month) | Secret retrieval and rotation polling |
| CloudWatch Logs | Interface | $0.01/hr (~$7.20/month) | ECS awslogs driver log delivery |
Five Interface endpoints cost ~$36/month but eliminate NAT data-processing charges for all AWS API calls. The break-even point depends on how frequently your tasks make API calls and pull images. Profile before adding all five — the ECR and SSM endpoints typically carry the most traffic. Start there, measure NAT data-processing charges for 1-2 weeks, then add the remaining endpoints if the savings justify the endpoint cost.
// CDK: S3 Gateway endpoint (free) + Interface endpoints
vpc.addGatewayEndpoint("S3Endpoint", {
service: ec2.GatewayVpcEndpointAwsService.S3,
});
for (const [id, service] of Object.entries({
EcrApi: ec2.InterfaceVpcEndpointAwsService.ECR,
EcrDkr: ec2.InterfaceVpcEndpointAwsService.ECR_DOCKER,
Ssm: ec2.InterfaceVpcEndpointAwsService.SSM,
SecretsManager: ec2.InterfaceVpcEndpointAwsService.SECRETS_MANAGER,
Logs: ec2.InterfaceVpcEndpointAwsService.CLOUDWATCH_LOGS,
})) {
new ec2.InterfaceVpcEndpoint(this, id + "Endpoint", {
vpc,
service,
subnets: { subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS },
securityGroups: [endpointSg],
privateDnsEnabled: true, // overrides public DNS — SDK calls route to endpoint automatically
});
}
VPC endpoint DNS requirement: privateDnsEnabled: true on each Interface endpoint makes the endpoint's private hostname the default resolution for that service. AWS SDK calls from your tasks automatically route to the private endpoint without any code changes. If you add a VPC endpoint but forget privateDnsEnabled, SDK calls still route through NAT — the endpoint exists but does nothing.
Pattern 2 — ALB configuration: IP target type, health checks, sticky sessions, idle timeout
The Application Load Balancer is the entry point for HTTP-transport and SSE-transport MCP servers on ECS Fargate. Four configuration decisions have disproportionate impact on reliability: the target type (which determines whether tasks register correctly), the health check endpoint (which determines how fast the ALB detects unhealthy tasks), sticky sessions (which determine whether SSE clients reconnect to the same task), and the idle timeout (which determines whether long-running tool calls succeed).
IP target type: the ECS awsvpc requirement
ECS Fargate with awsvpc networking gives each task its own Elastic Network Interface (ENI) and private IP address. The ALB registers tasks by their ENI IP, not by EC2 instance ID. If you create the target group with the default INSTANCE target type, tasks either cannot register or register but fail health checks permanently — ECS has no port mapping at the instance level for awsvpc tasks.
// CDK: ALB + target group — IP target type is mandatory for ECS awsvpc
import * as elbv2 from "aws-cdk-lib/aws-elasticloadbalancingv2";
const alb = new elbv2.ApplicationLoadBalancer(this, "McpAlb", {
vpc,
internetFacing: true,
securityGroup: albSg,
});
const targetGroup = new elbv2.ApplicationTargetGroup(this, "McpTargetGroup", {
vpc,
targetType: elbv2.TargetType.IP, // REQUIRED for ECS awsvpc networking
port: 3000,
protocol: elbv2.ApplicationProtocol.HTTP,
healthCheck: {
path: "/health",
interval: Duration.seconds(30),
timeout: Duration.seconds(5),
healthyThresholdCount: 2,
unhealthyThresholdCount: 3,
healthyHttpCodes: "200",
},
deregistrationDelay: Duration.seconds(30), // default is 300s — too slow for rolling deploys
stickinessCookieDuration: Duration.days(1), // enable sticky sessions for SSE/WebSocket
});
// HTTPS listener with ACM certificate
const httpsListener = alb.addListener("Https", {
port: 443,
protocol: elbv2.ApplicationProtocol.HTTPS,
certificates: [elbv2.ListenerCertificate.fromArn(certArn)],
defaultTargetGroups: [targetGroup],
});
// HTTP → HTTPS redirect
alb.addListener("Http", {
port: 80,
defaultAction: elbv2.ListenerAction.redirect({
protocol: "HTTPS",
port: "443",
permanent: true,
}),
});
Health check endpoint: dedicated route, no protocol overhead
The ALB health check must hit a path that responds immediately with HTTP 200, without triggering MCP protocol initialization, authentication checks, or database calls. The MCP server's main route is wrong for this — the handshake has initialization overhead and may return non-200 responses for protocol-negotiation requests. The ALB marks a target unhealthy after 3 consecutive failures at a 30-second interval (a 90-second window), causing task replacement churn if the main route is occasionally slow.
// Node.js: dedicated health endpoint — responds before any MCP session setup
import express from "express";
const app = express();
// Respond immediately with no dependencies
app.get("/health", (_req, res) => {
res.json({ status: "ok", timestamp: Date.now() });
});
// MCP SSE or Streamable HTTP endpoint — NOT used for health checks
app.get("/mcp", async (req, res) => {
// MCP transport setup here
});
app.listen(3000);
Two details matter: First, the health check route must be exempt from authentication middleware — the ALB sends a bare GET /health HTTP/1.1 with no auth headers, and any 401 or 403 marks the target unhealthy. Second, the route must respond within the health check timeout (5 seconds in the example above). If your server does any startup work (database connection pool warmup, config loading), the health endpoint should not wait for that work to complete — return 200 immediately and report readiness separately if needed.
Sticky sessions: SSE and WebSocket require pinned targets
MCP servers using Server-Sent Events or WebSocket transport maintain per-connection state in memory. If the ALB routes a client reconnection to a different task (which has no state for that session), the client sees a fresh event stream or a 404. Sticky sessions pin a client to the same target for the session duration.
| Transport | Sticky sessions needed? | Why |
|---|---|---|
| HTTP (stateless JSON-RPC) | No | Each request is independent; any task can handle any request |
| Streamable HTTP | Depends | If server tracks in-memory state per Mcp-Session-Id, yes; if state is in Redis/DB, no |
| SSE (legacy transport) | Yes | SSE is a persistent connection; reconnects must reach the same task |
| WebSocket | Yes (initial upgrade only) | ALB routes the upgrade request to one task; the TCP connection stays on that task until closed |
The ALB sets the AWSALB cookie on the first response. Subsequent requests from the same client include the cookie, and the ALB routes cookie-bearing requests to the same registered target. During rolling deploys, when a task is deregistered, the deregistrationDelay window (30 seconds in the example) allows in-flight requests to complete — but long-lived SSE connections on that task will be interrupted when the deregistration window expires. Design your MCP client to reconnect automatically when an SSE connection drops.
Idle timeout: long tool calls need extra headroom
The ALB has a default idle timeout of 60 seconds. MCP tool calls that involve long-running operations (database queries, API calls, AI inference) can exceed this limit. When the idle timeout is hit, ALB closes the connection and returns a 504 to the client. Two remedies: increase idleTimeout on the ALB (up to 4,000 seconds), or send SSE keepalive events every 30 seconds from the server to reset the idle timer.
// CDK: extend ALB idle timeout for long-running MCP tool calls
const alb = new elbv2.ApplicationLoadBalancer(this, "McpAlb", {
vpc,
internetFacing: true,
securityGroup: albSg,
idleTimeout: Duration.seconds(300), // extend from default 60s
});
// Server-side keepalive (Node.js SSE): send a comment event every 30s
// SSE comment events (": keepalive\n\n") reset the ALB idle timer
// without delivering a visible event to the client
setInterval(() => {
res.write(": keepalive\n\n");
}, 30_000);
Pattern 3 — CloudFront split: cached behaviors vs. pass-through, SSE and compress:false, WAF scope
CloudFront reduces ALB cost and latency for MCP servers that serve both static content (SEO pages, blog, landing page) and dynamic API or SSE streams. The critical insight is that CloudFront is a split system — it caches some paths and passes others through — and getting the split wrong either breaks caching for static content or buffers SSE streams, breaking live-event delivery.
Cache behavior split: two policies, two kinds of path
CloudFront evaluates behaviors in order from most-specific path pattern to the default (/*). Static content paths use CachingOptimized; API and MCP protocol paths use CachingDisabled with compress: false.
// CDK: CloudFront distribution with split cache behaviors
import * as cloudfront from "aws-cdk-lib/aws-cloudfront";
import * as origins from "aws-cdk-lib/aws-cloudfront-origins";
const distribution = new cloudfront.Distribution(this, "McpCdn", {
// Default behavior: pass through, no caching (covers /mcp, /api/*, unknown paths)
defaultBehavior: {
origin: new origins.LoadBalancerV2Origin(alb, {
protocolPolicy: cloudfront.OriginProtocolPolicy.HTTPS_ONLY,
}),
viewerProtocolPolicy: cloudfront.ViewerProtocolPolicy.REDIRECT_TO_HTTPS,
cachePolicy: cloudfront.CachePolicy.CACHING_DISABLED,
originRequestPolicy: cloudfront.OriginRequestPolicy.ALL_VIEWER_EXCEPT_HOST_HEADER,
allowedMethods: cloudfront.AllowedMethods.ALLOW_ALL,
compress: false, // never compress streaming responses
},
additionalBehaviors: {
"/seo/*": { // Static SEO pages: cache 24 hours
origin: new origins.LoadBalancerV2Origin(alb, {
protocolPolicy: cloudfront.OriginProtocolPolicy.HTTPS_ONLY,
}),
viewerProtocolPolicy: cloudfront.ViewerProtocolPolicy.REDIRECT_TO_HTTPS,
cachePolicy: cloudfront.CachePolicy.CACHING_OPTIMIZED,
compress: true,
},
"/blog/*": { // Blog posts: cache 24 hours
origin: new origins.LoadBalancerV2Origin(alb, {
protocolPolicy: cloudfront.OriginProtocolPolicy.HTTPS_ONLY,
}),
viewerProtocolPolicy: cloudfront.ViewerProtocolPolicy.REDIRECT_TO_HTTPS,
cachePolicy: cloudfront.CachePolicy.CACHING_OPTIMIZED,
compress: true,
},
"/mcp": { // MCP SSE/Streamable HTTP path: never cache, never compress
origin: new origins.LoadBalancerV2Origin(alb, {
protocolPolicy: cloudfront.OriginProtocolPolicy.HTTPS_ONLY,
readTimeout: Duration.seconds(60), // default is 30s — extend for long tool calls
}),
viewerProtocolPolicy: cloudfront.ViewerProtocolPolicy.REDIRECT_TO_HTTPS,
cachePolicy: cloudfront.CachePolicy.CACHING_DISABLED,
originRequestPolicy: cloudfront.OriginRequestPolicy.ALL_VIEWER_EXCEPT_HOST_HEADER,
allowedMethods: cloudfront.AllowedMethods.ALLOW_ALL,
compress: false, // CRITICAL — compression buffers the entire stream body
},
},
priceClass: cloudfront.PriceClass.PRICE_CLASS_100, // US + EU only; cheapest
});
Why compress:false on SSE is not optional
CloudFront's compress: true setting works by buffering the response body before compressing and forwarding it. For static HTML or JSON responses, this is fine. For SSE streams, this is fatal: CloudFront buffers the entire stream, and only delivers the complete buffered response to the client after the SSE connection closes — which is the opposite of a live-event stream. The client receives one burst of all events at the end (if it doesn't time out first), not a real-time stream.
| Protocol | CloudFront behavior | Required settings |
|---|---|---|
| HTML / JSON (static) | Caches and compresses at edge | cachePolicy: CACHING_OPTIMIZED, compress: true |
| SSE (Server-Sent Events) | Must pass through without buffering | cachePolicy: CACHING_DISABLED, compress: false, allowedMethods: ALLOW_ALL |
| WebSocket | Upgrades and proxies TCP connection | cachePolicy: CACHING_DISABLED, compress: false; CloudFront detects Upgrade: websocket automatically |
| Streamable HTTP | Pass through — same risk as SSE | Same as SSE — no buffering, no compression |
WAF scope: CloudFront WebACLs must be in us-east-1
This is one of the most confusing AWS constraints. CloudFront is a global service whose control plane lives in us-east-1. WAF WebACLs associated with CloudFront must be created with scope CLOUDFRONT in the us-east-1 region, regardless of where you're deploying everything else. A WebACL created in eu-west-1 or ap-southeast-1 with scope REGIONAL cannot be associated with a CloudFront distribution — attempting the association returns WAFInvalidParameterException.
// CDK: WAF for CloudFront must be in us-east-1 — use a cross-region stack
// In your CDK app:
const usEast1Stack = new cdk.Stack(app, "UsEast1Stack", { env: { region: "us-east-1" } });
const cloudfrontWebAcl = new wafv2.CfnWebACL(usEast1Stack, "CloudFrontWebAcl", {
scope: "CLOUDFRONT", // not "REGIONAL"
// ... rules
});
// WAF for ALB uses REGIONAL scope in your deployment region (any region)
const albWebAcl = new wafv2.CfnWebACL(this, "AlbWebAcl", {
scope: "REGIONAL",
// ... same rules
});
Cache invalidation on deploy
When you publish a new SEO page or blog post, the CloudFront cache at the edge still serves the old content (or returns 404 if the page is new) until the TTL expires or you invalidate the cache. The first 1,000 invalidation paths per month are free; additional paths cost $0.005 each. A wildcard path like /seo/* counts as a single path (not one per file in the directory), so prefer wildcard invalidations.
# Invalidate after deploying new SEO pages or blog posts
aws cloudfront create-invalidation \
--distribution-id DISTRIBUTION_ID \
--paths "/seo/*" "/blog/*" "/sitemap.xml" "/llms.txt"
# Targeted invalidation for a single page
aws cloudfront create-invalidation \
--distribution-id DISTRIBUTION_ID \
--paths "/seo/mcp-server-vpc" "/seo/mcp-server-alb"
In a CDK pipeline, add a CloudFormation custom resource that runs the invalidation as part of the deploy. In a shell-based deploy, add the invalidation command after the content push and before your deployment is considered complete.
Security response headers: one policy, all behaviors
CloudFront response headers policies add security headers to all matching responses at the edge, without touching application code. Apply the policy to every behavior, including the default. If you apply it only to static behaviors and forget the default, requests to /mcp or /api/* will lack the headers.
// CDK: security headers policy
const securityHeadersPolicy = new cloudfront.ResponseHeadersPolicy(this, "SecurityHeaders", {
securityHeadersBehavior: {
strictTransportSecurity: {
accessControlMaxAge: Duration.days(365),
includeSubdomains: true,
preload: true,
override: true,
},
xFrameOptions: {
frameOption: cloudfront.HeadersFrameOption.DENY,
override: true,
},
xContentTypeOptions: { override: true },
referrerPolicy: {
referrerPolicy: cloudfront.HeadersReferrerPolicy.STRICT_ORIGIN_WHEN_CROSS_ORIGIN,
override: true,
},
},
});
// Apply to EVERY behavior, not just static paths
// Pass responseHeadersPolicy to defaultBehavior and each entry in additionalBehaviors
Pattern 4 — Auth and security: Cognito access token validation, WAF rule tuning lifecycle, security headers
Authentication and WAF protection are the two active defense layers — they make decisions about individual requests rather than structuring the network. Cognito provides a managed OAuth 2.0 authorization server with per-tool scope granularity. WAF provides network-level protection against abuse and injection attacks. Both have non-obvious failure modes that create either security gaps or production outages.
Cognito: validate the access token, not the ID token
Cognito issues two JWTs after authentication: an ID token (containing user profile claims, intended for client-side display) and an access token (containing OAuth scopes, intended for API authorization). MCP servers must validate the access token. Three claims must be verified explicitly:
iss(issuer) — must match your specific user pool URLhttps://cognito-idp.REGION.amazonaws.com/POOL_ID. Without this check, a token from a different Cognito user pool would pass signature verification (AWS uses similar key infrastructure), allowing cross-tenant token acceptance.token_use— must be"access". Without this check, a client that accidentally sends the ID token instead of the access token will be accepted.- Custom scope string — Cognito's custom scopes include the resource server identifier as a prefix (
https://api.alivemcp.com/tools:read), not just the scope name (tools:read). Check the full prefixed string.
// CDK: Cognito user pool with resource server defining tool-level scopes
import * as cognito from "aws-cdk-lib/aws-cognito";
const userPool = new cognito.UserPool(this, "McpUserPool", {
selfSignUpEnabled: true,
signInAliases: { email: true },
removalPolicy: RemovalPolicy.RETAIN,
});
const resourceServer = userPool.addResourceServer("McpApiServer", {
identifier: "https://api.alivemcp.com",
scopes: [
{ scopeName: "tools:read", scopeDescription: "Call read-only MCP tools" },
{ scopeName: "tools:write", scopeDescription: "Call state-modifying MCP tools" },
{ scopeName: "admin", scopeDescription: "Full MCP server administration" },
],
});
const appClient = userPool.addClient("McpWebClient", {
generateSecret: false, // public client — use PKCE
oAuth: {
flows: { authorizationCodeGrant: true },
scopes: [
cognito.OAuthScope.OPENID,
cognito.OAuthScope.EMAIL,
cognito.OAuthScope.resourceServer(resourceServer, { scopeName: "tools:read", scopeDescription: "Read tools" }),
cognito.OAuthScope.resourceServer(resourceServer, { scopeName: "tools:write", scopeDescription: "Write tools" }),
],
callbackUrls: ["https://alivemcp.com/auth/callback"],
},
accessTokenValidity: Duration.hours(1),
refreshTokenValidity: Duration.days(30),
});
// Node.js: MCP server auth middleware using aws-jwt-verify
import { CognitoJwtVerifier } from "aws-jwt-verify";
const verifier = CognitoJwtVerifier.create({
userPoolId: process.env.COGNITO_USER_POOL_ID!,
tokenUse: "access", // MUST be "access" — reject ID tokens
clientId: process.env.COGNITO_CLIENT_ID!,
});
// aws-jwt-verify automatically caches JWKS for 1 hour and refreshes on unknown kid
export function requireScope(requiredScope: string) {
return async (req: Request, res: Response, next: NextFunction) => {
const authHeader = req.headers.authorization;
if (!authHeader?.startsWith("Bearer ")) {
return res.status(401).json({ error: "Missing Authorization header" });
}
const token = authHeader.slice(7);
try {
const payload = await verifier.verify(token);
// verifier has already checked: signature, iss, token_use, exp, aud/client_id
// Check custom scope — includes resource server identifier prefix
const scopes = (payload.scope as string)?.split(" ") ?? [];
if (!scopes.includes(`https://api.alivemcp.com/${requiredScope}`)) {
return res.status(403).json({ error: `Scope required: ${requiredScope}` });
}
(req as any).auth = payload;
next();
} catch {
return res.status(401).json({ error: "Invalid or expired token" });
}
};
}
app.use("/mcp", requireScope("tools:read"));
app.use("/mcp/admin", requireScope("admin"));
WAF rule tuning lifecycle: Count mode before Block
Never deploy WAF managed rule groups directly in Block mode on an MCP server. MCP payloads violate several common WAF assumptions:
SizeRestrictions_BODY— blocks request bodies over 8KB. An MCPtools/callrequest with a large document as context easily exceeds this.CrossSiteScripting_BODY— can block JSON payloads containing HTML fragments as tool arguments (e.g., a tool that processes web content).GenericRFI_BODY— can trigger on URL strings in tool arguments.
The correct workflow: deploy all managed rule groups with overrideAction: { count: {} } (Count mode) for one week. Query WAF logs for COUNTED requests on your MCP API paths. Rules that count zero legitimate requests are safe to switch to Block. Rules that count legitimate MCP requests must be added to excludedRules before switching.
// CDK: WAF WebACL for ALB — managed rules in Count mode initially
const webAcl = new wafv2.CfnWebACL(this, "McpWebAcl", {
scope: "REGIONAL",
defaultAction: { allow: {} },
visibilityConfig: {
cloudWatchMetricsEnabled: true,
metricName: "McpWafMetrics",
sampledRequestsEnabled: true,
},
rules: [
// Rate limit: block IPs exceeding 1000 requests/5 min on MCP/API paths
{
name: "RateLimitMcpApi",
priority: 1,
action: { block: {} },
visibilityConfig: { cloudWatchMetricsEnabled: true, metricName: "RateLimit", sampledRequestsEnabled: true },
statement: {
rateBasedStatement: {
limit: 1000,
aggregateKeyType: "IP",
scopeDownStatement: {
orStatement: {
statements: [
{ byteMatchStatement: { fieldToMatch: { uriPath: {} }, searchString: "/mcp", positionalConstraint: "STARTS_WITH", textTransformations: [{ priority: 0, type: "NONE" }] } },
{ byteMatchStatement: { fieldToMatch: { uriPath: {} }, searchString: "/api/", positionalConstraint: "STARTS_WITH", textTransformations: [{ priority: 0, type: "NONE" }] } },
],
},
},
},
},
},
// AWS Common Rule Set — COUNT mode initially, excluding rules known to false-positive on MCP
{
name: "AWSCommonRules",
priority: 2,
overrideAction: { count: {} }, // switch to { none: {} } after 1 week of log analysis
visibilityConfig: { cloudWatchMetricsEnabled: true, metricName: "CommonRules", sampledRequestsEnabled: true },
statement: {
managedRuleGroupStatement: {
vendorName: "AWS",
name: "AWSManagedRulesCommonRuleSet",
excludedRules: [
{ name: "SizeRestrictions_BODY" }, // MCP context payloads can exceed 8KB
{ name: "CrossSiteScripting_BODY" }, // JSON with HTML fragments in tool args
{ name: "GenericRFI_BODY" }, // URL strings in tool arguments
],
},
},
},
// Known Bad Inputs — safe to block immediately (no MCP false positives)
{
name: "KnownBadInputs",
priority: 3,
overrideAction: { none: {} }, // block mode is safe for this group
visibilityConfig: { cloudWatchMetricsEnabled: true, metricName: "BadInputs", sampledRequestsEnabled: true },
statement: {
managedRuleGroupStatement: {
vendorName: "AWS",
name: "AWSManagedRulesKnownBadInputsRuleSet",
},
},
},
],
});
// Associate with ALB
new wafv2.CfnWebACLAssociation(this, "WafAssociation", {
resourceArn: alb.loadBalancerArn,
webAclArn: webAcl.attrArn,
});
// WAF logging: bucket name must start with "aws-waf-logs-"
const wafLogsBucket = new s3.Bucket(this, "WafLogs", {
bucketName: `aws-waf-logs-mcp-${this.account}`,
lifecycleRules: [{ expiration: Duration.days(90) }],
blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL,
removalPolicy: RemovalPolicy.RETAIN,
});
new wafv2.CfnLoggingConfiguration(this, "WafLogging", {
resourceArn: webAcl.attrArn,
logDestinationConfigs: [wafLogsBucket.bucketArn],
redactedFields: [
{ singleHeader: { name: "authorization" } }, // redact auth tokens from WAF logs
{ singleHeader: { name: "cookie" } },
],
});
After one week in Count mode, query the WAF logs in Athena or CloudWatch Logs Insights for requests your API paths that were counted (not allowed) by each rule. If a rule counted zero requests on /mcp/* and /api/* paths, it's safe to switch to Block. Switch rules one at a time, watch the CloudWatch WAF metrics for unexpected Block spikes, then move to the next rule.
Deployment cycle: how all five layers wire together
Each layer depends on the ones below it. The correct build order in CDK or CloudFormation is:
- VPC — create the VPC, subnets, NAT gateway, and VPC endpoints. Security groups are created here but reference each other, so all three (
albSg,taskSg,endpointSg) must be created in the same stack. - Cognito user pool — create the user pool, resource server, and app client. The user pool ID and client ID are needed as environment variables in the ECS task definition.
- ACM certificate — request the certificate for your domain and validate it via DNS. This is a prerequisite for the ALB HTTPS listener. ACM certificates for CloudFront must be requested in
us-east-1(same constraint as CloudFront WAF). - WAF WebACL for ALB — create the REGIONAL WebACL in your deployment region. Do not associate it with the ALB yet if you're running in Count mode — the association can be done after the initial 1-week tuning window.
- WAF WebACL for CloudFront — create the CLOUDFRONT-scoped WebACL in
us-east-1. Cross-region CDK stacks require explicit environment specification. - ECS task definition + service — deploy the MCP server container. The task role needs IAM permissions to call SSM, Secrets Manager, and CloudWatch; the execution role needs ECR pull permissions.
- ALB + target group — create the ALB with the HTTPS listener pointing at the target group. ECS attaches the service to the target group. Associate the REGIONAL WAF WebACL.
- CloudFront distribution — create the distribution with the ALB as the origin. Associate the us-east-1 WAF WebACL. Add the security headers response policy.
The key constraint: CloudFront WAF and ACM certificate both require us-east-1. If you're deploying elsewhere, use cross-region CDK stacks for those resources and pass their ARNs into the main stack as context or SSM parameters.
// CDK app entry point: cross-region stack pattern
const app = new cdk.App();
// us-east-1 stack for CloudFront prerequisites
const globalStack = new GlobalStack(app, "GlobalStack", {
env: { region: "us-east-1" },
});
// Outputs: cloudfrontCertArn, cloudfrontWafWebAclArn
// Main deployment stack
const mainStack = new McpServerStack(app, "McpServerStack", {
env: { region: "eu-west-1" }, // or wherever you deploy
cloudfrontCertArn: globalStack.certArn,
cloudfrontWafWebAclArn: globalStack.webAclArn,
});
Consolidated failure mode reference
This table consolidates the most impactful failure modes across all five layers. These are the errors that are non-obvious at configuration time and expensive to diagnose after they surface in production.
| Layer | Symptom | Cause | Fix |
|---|---|---|---|
| VPC | CannotPullContainerError on ECS task startup |
Fargate task in private subnet has no NAT gateway and no ECR VPC endpoint | Add NAT gateway to a public subnet, or add ecr.api, ecr.dkr, and S3 Gateway endpoints |
| VPC | VPC endpoint exists but SDK still routes through NAT gateway (charges continue) | enableDnsHostnames or enableDnsSupport is false; privateDnsEnabled not set on endpoint |
Enable both VPC DNS options; set privateDnsEnabled: true on each Interface endpoint |
| VPC | Task can reach endpoint SG but SSM call returns 403 | VPC endpoint handles routing only — IAM still enforces; task role missing ssm:GetParametersByPath |
Add the required IAM action to the task role; VPC endpoints don't bypass IAM |
| ALB | All ECS targets show "unhealthy" in target group immediately after service creation | Target group uses INSTANCE type instead of IP for ECS awsvpc tasks |
Recreate target group with targetType: IP |
| ALB | Health check passes locally but fails in ALB | Health check path triggers auth middleware that returns 401 to ALB's source IP | Exempt the health check path from authentication middleware |
| ALB | SSE clients see blank event stream on reconnect | Sticky sessions not enabled; reconnect routed to different task with no session state | Enable stickinessCookieDuration on the target group, or externalize session state to Redis |
| ALB | ALB returns 504 after 60 seconds on long-running tool calls | ALB idle timeout (default 60s) exceeded by a slow tool execution | Increase ALB idleTimeout to 300s, or send SSE keepalive events (: keepalive\n\n) every 30s |
| CloudFront | SSE client receives one burst of all events then connection closes | CloudFront buffered the SSE body (compress: true or missing CACHING_DISABLED on SSE behavior) |
Set compress: false and cachePolicy: CACHING_DISABLED on the SSE path behavior |
| CloudFront | WAF WebACL association fails with "WAF web ACL not found" | WAF WebACL created in a region other than us-east-1 or with wrong scope |
Recreate WebACL with scope: CLOUDFRONT in us-east-1 |
| CloudFront | Deployed new SEO page but CloudFront still returns 404 or old content | Cache TTL not expired; no invalidation run after deploy | Run aws cloudfront create-invalidation --paths "/seo/*" after each deploy |
| Cognito | Token validation passes but scope check fails | Custom scope has resource server prefix (https://api.alivemcp.com/tools:read) but scope check only looks for tools:read |
Check the full prefixed scope string: scopes.includes("https://api.alivemcp.com/tools:read") |
| Cognito | Token from another tenant's user pool passes authentication | iss claim not verified — another Cognito pool's tokens pass signature check |
Configure verifier with your userPoolId; aws-jwt-verify verifies iss automatically |
| Cognito | Latency spike of 200ms on every authenticated request | JWKS fetched on every request instead of cached | Use aws-jwt-verify (caches JWKS automatically), or implement in-process LRU cache with 1-hour TTL |
| WAF | Large MCP tool call returns 403 with no application error log | SizeRestrictions_BODY rule blocks request body over 8KB before the request reaches the ALB |
Add SizeRestrictions_BODY to excludedRules in the CommonRuleSet statement |
| WAF | WAF logs not appearing in S3 bucket | Bucket name does not start with aws-waf-logs- (WAF enforces this prefix) |
Rename bucket or create a new one with the required prefix |
| WAF | WAF allows request but ALB still returns 403 | WAF allow does not override ALB security group or listener rules — evaluated independently | Check ALB security group allows traffic from CloudFront IP ranges or internet on port 443; WAF allow only means WAF didn't block |
The monitoring layer: where AliveMCP fits in
Each of the five layers has failure modes that are invisible until a real request fails. The VPC misconfiguration that routes SDK calls through NAT instead of endpoints shows up only as unexpectedly high data transfer costs. The ALB idle timeout that drops long tool calls shows up only when a user makes a slow tool call. The CloudFront SSE buffering issue shows up only when a client expects real-time events.
Production MCP server monitoring needs to test these layers end-to-end — not just ping the ALB health check from inside the VPC. AliveMCP runs external probes every 60 seconds against each registered MCP endpoint, verifying that the full stack (CloudFront → WAF → ALB → ECS task) responds correctly and within latency thresholds. When a probe fails, you know before your users do — and the failure detail (TLS handshake timeout vs. 403 from WAF vs. 504 from ALB) points directly at which layer to investigate.