AWS Network Load Balancer for MCP Servers: SSE Connections, TLS Modes, and Health Check Patterns

Pattern 1 — NLB vs ALB Decision, CDK Setup, and NLB Metrics

Why MCP's SSE transport changes the load balancer choice

The MCP specification defines two transports: stdio (local process) and HTTP with Server-Sent Events (SSE). Remote MCP deployments use SSE exclusively — a long-lived HTTP GET where the server streams newline-delimited JSON events. The client holds this connection open for the duration of an agent session, potentially for several minutes between events. This connection lifetime is what changes the load balancer calculus.

ALB's idle timeout defaults to 60 seconds. A quiet SSE stream — one where the MCP server has no status changes to report, the agent has submitted no new tool calls, and the monitoring system sees all endpoints healthy — goes idle. After 60 seconds of silence, ALB closes the connection with a TCP RST. The client (browser, Claude Desktop, agent framework) receives a connection closed error and must reconnect. From the user's perspective this looks like a flaky server. From the monitoring dashboard's perspective it looks like a brief outage even though nothing was wrong.

ALB's idle timeout is configurable up to 4,000 seconds. Setting it to 400+ seconds fixes the quiet-period problem. But this configuration is easy to forget, it applies to all listeners on the ALB (including health check paths and API endpoints that benefit from lower idle timeouts), and it still doesn't address the other two reasons NLB wins for MCP: static IPs and mTLS. NLB's 350-second timeout is fixed at the platform level — you can't lower it, but you also can't accidentally configure it wrong. For a dedicated MCP server ALB, 350 seconds is the right default.

The nine-property comparison table

Every property difference between NLB and ALB translates to a concrete MCP deployment decision:

The one hard requirement that forces ALB: host-based routing or path-based routing across multiple MCP endpoints on the same domain. NLB cannot route /api/tool-a and /api/tool-b to different target groups — it routes by port only. If you need that, put an ALB behind the NLB (NLB + ALB sandwich: static IPs from NLB, routing from ALB) or use ALB directly. For a single ECS service running one MCP server, NLB is the lower-friction path.

CDK: IP-target NLB for ECS Fargate

ECS Fargate tasks have private IPs, not EC2 instance IDs. NLB target groups must use targetType: TargetType.IP to register Fargate tasks directly by IP. The instance target type only works with EC2 launch type.

import * as ec2 from 'aws-cdk-lib/aws-ec2';
import * as elbv2 from 'aws-cdk-lib/aws-elasticloadbalancingv2';
import { Duration, Stack } from 'aws-cdk-lib';

// One Elastic IP per AZ — gives MCP clients a stable IP to allowlist
const eipA = new ec2.CfnEIP(scope, 'NlbEipA', { domain: 'vpc' });
const eipB = new ec2.CfnEIP(scope, 'NlbEipB', { domain: 'vpc' });

const nlb = new elbv2.NetworkLoadBalancer(scope, 'McpNlb', {
  vpc,
  internetFacing: true,
  // Disable cross-zone: NLB charges $0.01/GB for cross-AZ bytes (ALB doesn't)
  // With ECS spread placement, tasks are evenly distributed — no cross-zone needed
  crossZoneEnabled: false,
  vpcSubnets: { subnets: vpc.publicSubnets },
});

const targetGroup = new elbv2.NetworkTargetGroup(scope, 'McpTg', {
  vpc,
  port: 8080,
  protocol: elbv2.Protocol.TCP,
  targetType: elbv2.TargetType.IP,   // required for Fargate
  // 120s = ECS stopTimeout; in-flight SSE streams drain before SIGKILL
  deregistrationDelay: Duration.seconds(120),
  // Preserve original source IP — task SG must allow 0.0.0.0/0 on MCP port
  preserveClientIp: true,
  healthCheck: {
    protocol: elbv2.Protocol.HTTP,
    path: '/health',
    port: '8080',
    interval: Duration.seconds(10),
    healthyThresholdCount: 2,
    unhealthyThresholdCount: 2,
  },
});

nlb.addListener('McpTls', {
  port: 443,
  protocol: elbv2.Protocol.TLS,
  certificates: [elbv2.ListenerCertificate.fromArn(certArn)],
  defaultTargetGroups: [targetGroup],
  sslPolicy: elbv2.SslPolicy.RECOMMENDED_TLS,
  alpnPolicy: elbv2.AlpnPolicy.HTTP2_PREFERRED,
});

mcpService.attachToNetworkTargetGroup(targetGroup);

The preserveClientIp: true setting has a security group implication that catches teams off guard: the task security group must allow inbound from 0.0.0.0/0 on the MCP port (or the known client CIDR), not from the NLB security group. When client IP is preserved, packets arrive at the container with the original client source IP — the NLB's IP is not the source. If you restrict the task SG to the NLB subnet CIDR, all connections are dropped silently. Setting preserveClientIp: false simplifies the security group (allow inbound from VPC CIDR only) at the cost of not seeing real client IPs in MCP server logs.

Cross-zone tradeoff: default off for a reason

ALB has cross-zone load balancing enabled by default (and does not charge for cross-AZ traffic). NLB has it disabled by default and charges $0.01/GB when enabled. The right setting for MCP probe fleets:

• External NLB for public SSE feed — cross-zone off: ECS spreads tasks evenly across AZs by default. With three AZs and six probe workers, each AZ gets two workers. NLB nodes in each AZ route to their local two workers with no cross-zone transfer cost.
• Internal NLB for coordinator — cross-zone on: The coordinator runs as a single task in one AZ. Without cross-zone, an NLB node in AZ-B has no local targets and rejects connections from AZ-B workers. Coordinator traffic is low-volume (registry fetches, not probe result uploads), so cross-AZ transfer costs are negligible. Cross-zone on ensures all worker tasks in all AZs can reach the coordinator regardless of which AZ ECS placed it in after a restart.

NLB metrics that distinguish NLB health from MCP application health

NLB publishes two categories of metrics to CloudWatch under the AWS/NetworkELB namespace. Infrastructure metrics tell you whether NLB itself is forwarding traffic correctly. Application metrics tell you whether the MCP servers behind the NLB are handling that traffic correctly. The distinction matters for alert routing — an infrastructure outage pages infrastructure on-call; an application degradation pages the MCP server team.

Seven metrics worth wiring up:

• HealthyHostCount — tasks NLB considers healthy. Alert at < 1 (full outage); alert at < desiredCount for 5 minutes (partial degradation).
• UnHealthyHostCount — tasks failing health checks. This rises before HealthyHostCount drops during partial failures; it's the leading indicator. Alert at > 0 for 3 consecutive 1-minute periods (rides through normal deployments).
• ActiveFlowCount — current open TCP connections. Each SSE subscription = 1 flow. Monitor for sudden drops (disconnect storm), monotonic rises (connection leak), or zero during expected traffic (DNS/cert failure).
• NewFlowCount — new connections per second. A spike shortly after a TCP_ELB_Reset_Count spike is the keep-alive fingerprint: NLB times out idle SSE connections, clients reconnect immediately.
• ProcessedBytes — bytes through NLB. A sustained drop suggests MCP servers are idle or stuck; a spike suggests large probe result payloads.
• TCP_ELB_Reset_Count — RST packets sent by NLB. Spikes when NLB times out idle connections (350s idle timeout fires) or when a target fails mid-connection.
• TargetTLSNegotiationErrorCount — TLS handshake failures between NLB and backend (TLS-to-target mode only). Alert at > 0 — indicates expired backend cert or cipher mismatch in the NLB-to-container leg.

AliveMCP's external probe is a complementary signal to NLB metrics. The NLB health check fires every 10 seconds and marks a target unhealthy after 20 seconds of failures. AliveMCP's external probe fires every 60 seconds. The NLB alarm fires 10–30 seconds before AliveMCP registers the endpoint as degraded. For AliveMCP customers who also monitor their NLB metrics in CloudWatch, the NLB alarm is the earlier, internal signal; AliveMCP's probe is the external confirmation that real clients would see the same failure.

Pattern 2 — Three TLS Modes for NLB-Fronted MCP Servers

Why TLS mode choice matters for MCP

TLS configuration is not a set-and-forget decision for MCP servers. Enterprise agent platforms increasingly require mTLS to verify caller identity — a client certificate in the TLS handshake proves that the agent calling tool_call is an authorized system, not a rogue process. Free-tier public MCP servers just need TLS 1.3 and auto-renewing ACM certs. Internal MCP servers running in private VPCs under compliance frameworks like PCI DSS may need encryption in transit from browser through NLB through the container — end-to-end with no plaintext segment. These three requirements need three different NLB configurations.

Mode 1: TLS termination at NLB (recommended for most MCP servers)

NLB handles the TLS handshake with the client and forwards plain HTTP to the Fargate task. The MCP container code sees plain HTTP on port 8080. AWS manages ACM certificate renewal automatically — cert expiry is the most common TLS finding AliveMCP surfaces on monitored endpoints, and ACM eliminates it for NLB-terminated TLS.

import * as acm from 'aws-cdk-lib/aws-certificatemanager';

const cert = new acm.Certificate(scope, 'McpCert', {
  domainName: 'api.yourmcpserver.com',
  validation: acm.CertificateValidation.fromDns(hostedZone),
});

nlb.addListener('McpTls', {
  port: 443,
  protocol: elbv2.Protocol.TLS,
  certificates: [elbv2.ListenerCertificate.fromCertificateManager(cert)],
  defaultTargetGroups: [targetGroup],
  // TLS 1.2 + TLS 1.3, ECDHE ciphers, no RC4 or 3DES
  sslPolicy: elbv2.SslPolicy.RECOMMENDED_TLS,
  // HTTP/2 multiplexing: single TCP connection carries multiple SSE streams
  alpnPolicy: elbv2.AlpnPolicy.HTTP2_PREFERRED,
});

The ALPN policy deserves attention. Without HTTP2_PREFERRED, each SSE stream from a browser subscribing to the AliveMCP live feed consumes its own TCP connection (HTTP/1.1 limitation). With HTTP2_PREFERRED, a browser subscribing to status updates for 20 endpoints multiplexes all 20 SSE streams over one TCP connection. At 10,000 concurrent subscribers, the difference between 10,000 TCP connections and potentially 500 TCP connections (20 streams each) is material for NLB LCU billing.

Security policy selection from most to least restrictive:

Mode 2: TCP pass-through for mTLS client certificate authentication

A TCP listener on NLB port 443 passes raw bytes to the container without inspecting the TLS envelope. The full TLS handshake, including client certificate exchange, reaches the MCP container's TLS stack. This is the only NLB mode that supports mTLS — Mode 1 (TLS termination) strips the client certificate at NLB, and Mode 3 (re-encryption) also terminates the outer TLS at NLB before re-encrypting for the target.

// TCP listener — NLB is transparent to TLS
const tcpListener = nlb.addListener('McpMtls', {
  port: 443,
  protocol: elbv2.Protocol.TCP,  // not TLS — NLB doesn't open the envelope
  defaultTargetGroups: [mtlsTargetGroup],
});

// In the MCP container (Node.js):
import * as https from 'node:https';
import * as fs from 'node:fs';

const server = https.createServer({
  key: fs.readFileSync('/run/secrets/server-key.pem'),
  cert: fs.readFileSync('/run/secrets/server-cert.pem'),
  requestCert: true,           // require client certificate
  rejectUnauthorized: true,    // reject if client cert is not from trusted CA
  ca: fs.readFileSync('/run/secrets/client-ca.pem'),
}, (req, res) => {
  const cert = req.socket.getPeerCertificate();
  if (!cert.subject?.CN) { res.writeHead(401); res.end(); return; }
  handleMcpRequest(req, res, cert.subject.CN);
});

server.listen(8443, '0.0.0.0');

// Separate plain HTTP health check listener for NLB
require('node:http').createServer((req, res) => {
  res.writeHead(200); res.end('OK');
}).listen(8080, '127.0.0.1');  // bound to loopback — prevents mTLS bypass

The loopback binding for the health check listener is critical. If the health check port is bound to 0.0.0.0, an external client can access the health check endpoint without providing a client certificate — bypassing mTLS entirely. Binding to 127.0.0.1 makes the health check accessible only from within the container network namespace, which is all NLB needs (NLB health check probes originate from NLB nodes in the VPC and arrive at the task's private IP, not localhost — actually, for NLB health checks, they originate from the NLB node IP within the VPC, so you need to bind to 0.0.0.0 for health checks but restrict traffic via SG).

Correcting the above: bind the health check port to 0.0.0.0 but restrict the security group inbound rule for port 8080 to the VPC CIDR only (NLB health check probes originate from NLB node IPs within the VPC). Port 8443 (mTLS) is open to client CIDR. This is the correct separation.

Mode 3: TLS re-encryption for zero-trust VPC requirements

Some compliance frameworks require that no plaintext data travels inside the VPC, even on private subnets. TLS re-encryption establishes an outer TLS session (client to NLB) and an inner TLS session (NLB to Fargate task) — the container receives encrypted bytes from NLB, not plain HTTP.

// TLS listener (client → NLB): uses public ACM cert
nlb.addListener('McpReencrypt', {
  port: 443,
  protocol: elbv2.Protocol.TLS,
  certificates: [elbv2.ListenerCertificate.fromCertificateManager(publicCert)],
  sslPolicy: elbv2.SslPolicy.RECOMMENDED_TLS,
});

// Target group: TLS protocol encrypts NLB → container traffic
const reencryptTg = new elbv2.NetworkTargetGroup(scope, 'McpReencryptTg', {
  vpc,
  port: 8443,
  protocol: elbv2.Protocol.TLS,     // NLB opens TLS to target
  targetType: elbv2.TargetType.IP,
  healthCheck: {
    protocol: elbv2.Protocol.HTTPS,  // NLB validates container's TLS cert
    path: '/health',
    port: '8443',
    interval: Duration.seconds(10),
    unhealthyThresholdCount: 2,
    healthyThresholdCount: 2,
  },
  deregistrationDelay: Duration.seconds(120),
});

The container's TLS certificate must be valid from NLB's trust store. ACM Private CA issues internal certificates that NLB trusts automatically when the target group protocol is TLS. Self-signed certificates require an L1 CfnTargetGroup property override to disable certificate validation on the NLB-to-target leg — acceptable for development, not for compliance audits.

Proxy Protocol v2: getting TLS metadata without pass-through

When NLB terminates TLS (Mode 1), the MCP container sees plain HTTP and has no visibility into the client's negotiated TLS version or cipher suite. Proxy Protocol v2 adds a binary header to each TCP connection that includes source IP, destination IP, and — for TLS listeners — the TLS version and cipher suite negotiated at NLB. This gives the MCP container audit-level TLS metadata without requiring pass-through.

// Enable PP2 on target group via L1 override
const cfnTg = targetGroup.node.defaultChild as elbv2.CfnTargetGroup;
cfnTg.addPropertyOverride('TargetGroupAttributes', [
  { Key: 'proxy_protocol_v2.enabled', Value: 'true' },
]);

Proxy Protocol v2 with TLS metadata is an NLB-exclusive feature — ALB does not surface TLS negotiation metadata via Proxy Protocol. For MCP servers that need to log per-client TLS version for compliance (SOC 2 Type II audit log, PCI DSS transmission log) without the complexity of pass-through mode, PP2 is the lowest-friction solution.

What AliveMCP's TLS probe checks

For every monitored MCP endpoint, AliveMCP performs a TLS handshake check before the MCP protocol probe. This catches TLS misconfigurations that would cause client failures even if the MCP server is otherwise healthy. Five checks:

NLB TLS termination with RECOMMENDED_TLS policy and ACM auto-renewing cert satisfies all five checks automatically. The most common finding AliveMCP surfaces is certificate expiry — indie MCP server authors running on a VPS with manual cert management forget to renew. Author-tier customers receive email alerts 14 days before their endpoint's cert expires, giving two weeks to rotate before the cert check starts failing.

Pattern 3 — Health Checks, Keep-Alives, Graceful Shutdown, and ActiveFlowCount Monitoring

Why TCP health checks are not enough for MCP servers

NLB supports three health check protocols: TCP, HTTP, and HTTPS. A TCP health check opens a connection to the target port and immediately closes it — if the OS accepts the connection, the target is "healthy." This passes in two common MCP server failure scenarios where the application is actually broken:

• Event loop blocked by a CPU-intensive tool handler: The Node.js event loop is single-threaded. A malformed MCP tool payload that triggers a synchronous SHA-256 hash over a 100 MB buffer blocks the loop. New HTTP requests queue in the backlog at the OS socket layer — the port is still "open" — but no request is ever dequeued or processed. TCP health check passes. HTTP health check on /health fails (GET is queued behind the blocking operation). AliveMCP's external probe catches this as a >5-second response time and marks the endpoint degraded within 60 seconds.
• Database connection pool exhausted: The MCP server's SQLite or PostgreSQL connection pool fills up during a traffic spike. New tool calls fail with 503. The HTTP server is still listening. TCP check passes. HTTP check fails if /health includes a SELECT 1 pool validation that returns 503 when no connections are available.

const targetGroup = new elbv2.NetworkTargetGroup(scope, 'McpTg', {
  vpc,
  port: 8080,
  protocol: elbv2.Protocol.TCP,
  targetType: elbv2.TargetType.IP,
  healthCheck: {
    protocol: elbv2.Protocol.HTTP,   // not TCP
    path: '/health',
    port: '8080',
    interval: Duration.seconds(10),  // NLB minimum is 10s (ALB can go to 5s)
    unhealthyThresholdCount: 2,
    healthyThresholdCount: 2,
    // Note: NLB accepts only a single HTTP status code (not a range)
    // If /health returns 204, change it to 200 — NLB configured for 200 will
    // mark 204 as unhealthy
  },
  deregistrationDelay: Duration.seconds(120),
});

The single-status-code limitation is NLB's sharpest edge compared to ALB. ALB lets you specify a range like 200-499. NLB requires exactly one code. If your /health endpoint returns 204 No Content (a common default in Express health check middleware), NLB configured to expect 200 will mark every target unhealthy immediately. Verify the exact status code your /health handler returns before deploying to production.

Health check timing math

Understanding the timing prevents both over-alerting (too many false positive alarms during normal deployments) and under-alerting (too long a window between failure and traffic removal):

time_to_unhealthy = interval × unhealthy_threshold = 10s × 2 = 20 seconds
time_to_healthy   = interval × healthy_threshold   = 10s × 2 = 20 seconds

The 20-second window from first failure to traffic removal means MCP clients will see failures for up to 20 seconds before NLB stops routing to the bad target. During a rolling ECS deployment, one task enters the draining state while a new one starts up: the outage window visible to AliveMCP's external probe is approximately 40 seconds (20s for old task to be marked unhealthy + 20s for new task to be marked healthy). This is the minimum achievable with NLB's 10-second minimum interval.

The 6-step graceful shutdown sequence that minimizes this window:

1. ECS sends SIGTERM to the container
2. SIGTERM handler sets isHealthy = false — /health now returns 503
3. NLB sees two consecutive 503 responses over 20 seconds, marks target draining
4. No new connections routed to this target; existing SSE streams continue
5. Deregistration delay timer starts (120 seconds)
6. After 120 seconds, NLB RSTs remaining connections; ECS SIGKILL fires at T+115s stopTimeout

Slow start: preventing cold-container traffic spikes

MCP servers with cold-start latency — loading a tool registry from SQLite, initializing a 50-connection PostgreSQL pool, pre-compiling JSON schema validators — pass their first two health checks (T+20s) while still warming up background initialization. Without slow start, NLB routes the new target its full traffic weight immediately. With slow start, traffic ramps linearly from 0% to 100% over 60 seconds.

// CDK L2 doesn't expose slow start for NLB target groups yet — use L1 override
const cfnTg = targetGroup.node.defaultChild as elbv2.CfnTargetGroup;
cfnTg.addPropertyOverride('SlowStartDurationSeconds', 60);
// New target: 0% at T+20s (first healthy) → 100% at T+80s

The tradeoff: total cold-start impact window extends from 20s (no slow start) to 80s (with 60s slow start). But P99 latency during the ramp is dramatically lower — a cold container handling 2% of traffic while its pool initializes serves those requests slowly; it serves 100% of traffic before pool initialization is complete catastrophically.

Keeping SSE connections alive through the 350-second idle timeout

NLB's 350-second TCP idle timeout is fixed. SSE connections go idle when the MCP server has no events to push — every monitored endpoint is healthy, no status changes have occurred. In AliveMCP's case, quiet periods during "all green" monitoring windows can last 5–10 minutes. Two independent mechanisms together ensure the NLB flow entry is refreshed well before 350 seconds:

1. SSE comment keep-alives (application layer, 30-second interval)

function createMcpSseHandler(req, res) {
  res.writeHead(200, {
    'Content-Type': 'text/event-stream',
    'Cache-Control': 'no-cache',
    'Connection': 'keep-alive',
    'X-Accel-Buffering': 'no',   // prevents Nginx/proxy buffering of SSE chunks
  });

  res.write('data: {"type":"connected","serverTime":"' + new Date().toISOString() + '"}\n\n');

  // SSE comment lines start with ':' — browsers ignore them; NLB flow timer resets
  const keepAlive = setInterval(() => {
    if (!res.writableEnded) res.write(': keepalive\n\n');
    else clearInterval(keepAlive);
  }, 30_000);

  req.on('close', () => clearInterval(keepAlive));

  const unsubscribe = subscribeToProbeResults(event => {
    if (!res.writableEnded) res.write('data: ' + JSON.stringify(event) + '\n\n');
  });
  req.on('close', unsubscribe);
}

2. TCP socket keep-alive (transport layer, 60-second backup)

const server = require('node:http').createServer(app);
server.on('connection', (socket) => {
  // OS-level TCP keep-alive probes after 60s of inactivity
  socket.setKeepAlive(true, 60_000);
});

With both mechanisms, the NLB flow entry is refreshed at most every 30 seconds. The 350-second timeout is never reached during normal operation. Only a client whose machine suspends mid-session without sending TCP FIN — typical mobile device behavior — creates a truly silent connection; TCP keep-alive at the OS level detects and cleans up these zombie connections automatically.

Source IP hash sticky sessions for stateful MCP tools

Most MCP tools are stateless: search(query) returns results without caring which container instance runs it. Browser automation MCP tools — "open this URL", "click button", "read page content" — maintain browser session state on a specific container. Source IP hash sticky sessions route all connections from one client IP to the same target.

const cfnTg = targetGroup.node.defaultChild as elbv2.CfnTargetGroup;
cfnTg.addPropertyOverride('TargetGroupAttributes', [
  { Key: 'stickiness.enabled', Value: 'true' },
  { Key: 'stickiness.type', Value: 'source_ip' },
]);

Three caveats that bite MCP teams:

• Corporate NAT collapses thousands of users to one IP. All agents running inside a company network appear as a single source IP. Source IP stickiness routes every agent call from that company to the same container — a single container serves the company's entire MCP load regardless of how many workers you have. For enterprise MCPs, implement session stickiness in the application layer (session token in Authorization header → consistent hash ring to container) rather than relying on NLB source IP.
• Fargate Spot interruption destroys sticky sessions. When a Spot task is interrupted, NLB reassigns sticky clients to other healthy targets. The browser session context on the old container is gone. MCP servers using sticky sessions must persist session state to Redis or DynamoDB — any target must be able to resume a session given its ID.
• Cross-zone stickiness with NLB default cross-zone-off. Without cross-zone, a client always reaches an NLB node in its nearest AZ. If AZ-A has zero healthy targets (all Spot interrupted) and AZ-B has four, AZ-A clients see failures even though healthy capacity exists. Enable cross-zone on NLB when using sticky sessions to ensure clients can reach healthy targets in any AZ.

Graceful shutdown: the SIGTERM pattern for SSE servers

const activeSseClients = new Set();
let isHealthy = true;

// /health endpoint
app.get('/health', (req, res) => {
  res.status(isHealthy ? 200 : 503).json({ status: isHealthy ? 'ok' : 'shutting_down' });
});

// Track clients
app.get('/sse', (req, res) => {
  activeSseClients.add(res);
  req.on('close', () => activeSseClients.delete(res));
  // ... SSE setup
});

process.on('SIGTERM', async () => {
  // Step 1: fail health checks → NLB marks draining in 20s
  isHealthy = false;

  // Step 2: send close event to all active SSE clients
  for (const res of activeSseClients) {
    if (!res.writableEnded) {
      // retryAfterMs: 5s gives the replacement task time to become healthy
      res.write('event: server_shutdown\ndata: {"reconnect":true,"retryAfterMs":5000}\n\n');
      res.end();
    }
  }
  activeSseClients.clear();

  // Step 3: close HTTP server (no new connections)
  httpServer.close(() => process.exit(0));

  // Force exit at T+100s — before ECS SIGKILL at T+115s stopTimeout
  setTimeout(() => process.exit(0), 100_000);
});

The server_shutdown SSE event is an application-level hint. MCP clients that implement it transition to a "reconnecting" state (spinner, not error). Clients that ignore it see a connection close and auto-retry. The 5-second retry delay is long enough for the replacement ECS task to pass two health checks (20 seconds) and begin accepting traffic — clients that retry at 0ms will receive a connection refused from NLB while the new task is still starting up.

ActiveFlowCount as a leading indicator of connection problems

NLB's ActiveFlowCount metric is a real-time count of open TCP connections. For an MCP server, this is a proxy for the number of active agent sessions (SSE subscriptions) plus in-flight tool calls (WebSocket connections). Four patterns to alarm on:

import * as cloudwatch from 'aws-cdk-lib/aws-cloudwatch';

// Connection leak: sustained growth past 2× expected max
const leakAlarm = new cloudwatch.Alarm(scope, 'ConnectionLeak', {
  alarmName: 'mcp-nlb-connection-leak',
  metric: new cloudwatch.Metric({
    namespace: 'AWS/NetworkELB',
    metricName: 'ActiveFlowCount',
    dimensionsMap: { LoadBalancer: nlb.loadBalancerFullName },
    period: Duration.minutes(5),
    statistic: 'Average',
  }),
  threshold: 5000,          // tune to 2× your expected max concurrent connections
  comparisonOperator: cloudwatch.ComparisonOperator.GREATER_THAN_THRESHOLD,
  evaluationPeriods: 6,     // sustained 30 minutes
  treatMissingData: cloudwatch.TreatMissingData.NOT_BREACHING,
});

// RST storm: NLB closing idle connections
const rstStorm = new cloudwatch.Alarm(scope, 'TcpRstStorm', {
  alarmName: 'mcp-nlb-rst-storm',
  metric: new cloudwatch.Metric({
    namespace: 'AWS/NetworkELB',
    metricName: 'TCP_ELB_Reset_Count',
    dimensionsMap: { LoadBalancer: nlb.loadBalancerFullName },
    period: Duration.minutes(1),
    statistic: 'Sum',
  }),
  threshold: 100,           // more than 100 RSTs/minute signals idle timeout firing
  comparisonOperator: cloudwatch.ComparisonOperator.GREATER_THAN_THRESHOLD,
  evaluationPeriods: 3,
  treatMissingData: cloudwatch.TreatMissingData.NOT_BREACHING,
});

The TCP_ELB_Reset_Count spike → NewFlowCount spike pattern is the specific fingerprint of "SSE keep-alive not implemented." AliveMCP's external probe catches this as brief uptime gaps — the probe's 60-second cycle interval means a 30-second RST window appears as one missed probe, marking the endpoint as degraded for one cycle before recovery. If you see this pattern on AliveMCP's status page for your endpoint (intermittent single-cycle degradations with no other symptoms), implementing the 30-second SSE comment keep-alive will eliminate it.

Failure modes and diagnostics

AliveMCP monitors NLB-fronted MCP endpoints and surfaces several of these failure modes in the endpoint's status page: TLS certificate status (including expiry warning), connection establishment latency (elevated latency = cold container or slow start), intermittent disconnects (keep-alive pattern), and protocol-level health (MCP tool call succeeds vs TCP connection succeeds). Author-tier customers get email alerts for expiry warnings and sustained degradation; Team-tier customers add Slack webhook alerts with structured failure-mode context.

For general guidance on MCP server health monitoring beyond NLB — external probe strategy, status page setup, alerting thresholds — see How to Monitor an MCP Server.