ECS Deployment Circuit Breaker for MCP Servers

Enabling the deployment circuit breaker

The circuit breaker is a property of the ECS service, not the task definition. Enable it when creating or updating the service:

aws ecs create-service \
  --cluster my-cluster \
  --service-name mcp-server-svc \
  --task-definition mcp-server:5 \
  --desired-count 3 \
  --launch-type FARGATE \
  --network-configuration "awsvpcConfiguration={subnets=[subnet-abc],securityGroups=[sg-abc],assignPublicIp=DISABLED}" \
  --load-balancers "targetGroupArn=arn:aws:elasticloadbalancing:us-east-1:123456789012:targetgroup/mcp-server-tg/abc,containerName=mcp-server,containerPort=3000" \
  --deployment-configuration '{
    "deploymentCircuitBreaker": {
      "enable": true,
      "rollback": true
    },
    "maximumPercent": 200,
    "minimumHealthyPercent": 100
  }' \
  --health-check-grace-period-seconds 60

Updating an existing service to add the circuit breaker:

aws ecs update-service \
  --cluster my-cluster \
  --service mcp-server-svc \
  --deployment-configuration '{
    "deploymentCircuitBreaker": { "enable": true, "rollback": true },
    "maximumPercent": 200,
    "minimumHealthyPercent": 100
  }' \
  --health-check-grace-period-seconds 60

In CDK:

const service = new ecs.FargateService(this, 'McpServerService', {
  cluster,
  taskDefinition,
  desiredCount: 3,
  circuitBreaker: { rollback: true },
  deploymentController: { type: ecs.DeploymentControllerType.ECS },
  healthCheckGracePeriod: Duration.seconds(60),
  minHealthyPercent: 100,
  maxHealthyPercent: 200,
});

How the circuit breaker threshold works

The circuit breaker monitors the deployment and trips when a threshold of consecutive failed tasks is reached. ECS calculates the threshold from the service's desired count:

A task "fails" when ECS marks it as stopped due to health check failure. If your MCP server task starts successfully but fails the ALB health check (incorrect /health path, wrong port, startup takes too long), the circuit breaker counts that as a failure and rolls back after the threshold is reached.

DeploymentConfiguration for zero-downtime rolling deployments

Two parameters control how many tasks ECS replaces simultaneously during a deployment:

--deployment-configuration '{
  "maximumPercent": 200,
  "minimumHealthyPercent": 100
}'

For a 4-task MCP server service with maximumPercent: 200, minimumHealthyPercent: 100, a deployment proceeds like this:

Desired: 4 tasks

Step 1: Start 4 new tasks  → 8 tasks running (4 old + 4 new)
Step 2: New tasks pass health check and register with ALB
Step 3: Stop 4 old tasks   → 4 tasks running (4 new only)

At no point are fewer than 4 healthy tasks serving traffic.

Avoid minimumHealthyPercent: 50 for production MCP servers — it allows ECS to stop half the running tasks before starting replacements, halving capacity during the deployment window.

Alarm-based rollback for application errors

The deployment circuit breaker only catches tasks that fail ECS health checks. It does not catch a deployment where tasks start successfully but the application code is broken — returning 500s, crashing on specific tool calls, or silently returning empty responses. Use alarm-based rollback to catch these application-level regressions:

# Create a CloudWatch alarm on ALB 5xx rate
aws cloudwatch put-metric-alarm \
  --alarm-name mcp-server-5xx-high \
  --metric-name HTTPCode_Target_5XX_Count \
  --namespace AWS/ApplicationELB \
  --dimensions \
      Name=LoadBalancer,Value=app/my-alb/abc123 \
      Name=TargetGroup,Value=targetgroup/mcp-server-tg/def456 \
  --statistic Sum \
  --period 60 \
  --threshold 10 \
  --comparison-operator GreaterThanOrEqualToThreshold \
  --evaluation-periods 2 \
  --treat-missing-data notBreaching

# Update the service to use alarm-based rollback
aws ecs update-service \
  --cluster my-cluster \
  --service mcp-server-svc \
  --deployment-configuration '{
    "deploymentCircuitBreaker": { "enable": true, "rollback": true },
    "alarms": {
      "alarmNames": ["mcp-server-5xx-high"],
      "enable": true,
      "rollback": true
    },
    "maximumPercent": 200,
    "minimumHealthyPercent": 100
  }'

With alarm-based rollback, ECS monitors the specified CloudWatch alarms throughout the deployment. If any alarm enters ALARM state during the deployment window, ECS triggers an automatic rollback to the previous task definition — even if all new tasks passed ECS health checks.

Recommended alarms to wire into deployment rollback for MCP servers:

Health check grace period

The health check grace period tells ECS to ignore ALB health check failures for the first N seconds after a task starts. Without it, the circuit breaker may fire on a valid new task that is still initializing:

# MCP server startup sequence that can take 30-60s:
# 1. Node.js process starts (0-2s)
# 2. Load and validate tool schemas from disk (2-10s)
# 3. Warm-up connections to downstream services (10-30s)
# 4. Register with MCP registry (30-60s, if enabled)
# → ALB health check at /health returns 503 during steps 1-4

# Without grace period: ECS marks the task unhealthy at step 1,
# circuit breaker trips after threshold tasks fail, rollback triggered.
# New deployment fails even though the code is correct.

# Correct configuration:
--health-check-grace-period-seconds 60

The grace period applies per-task, per-deployment. After the grace period expires, the ALB health check results determine whether the task is registered or deregistered from the target group. Set the grace period to the 95th percentile startup time of your MCP server, not the average — you want to protect against slow starts, not just typical starts.

Common failures