CloudFormation Change Sets for Safe MCP Server Deployments

Change set lifecycle

A change set is a separate API object that describes proposed changes. Creating it does not modify the stack. Only ExecuteChangeSet applies the changes:

# Step 1 — create the change set
aws cloudformation create-change-set \
  --stack-name mcp-server-prod \
  --change-set-name deploy-2026-09-26 \
  --template-body file://template.yaml \
  --parameters ParameterKey=Environment,ParameterValue=production \
               ParameterKey=ImageTag,ParameterValue=sha-abc1234 \
  --capabilities CAPABILITY_IAM

# Step 2 — poll until CREATE_COMPLETE (or FAILED)
aws cloudformation describe-change-set \
  --stack-name mcp-server-prod \
  --change-set-name deploy-2026-09-26 \
  --query "Status"

# Step 3 — inspect the changes (look for Replacement or Remove)
aws cloudformation describe-change-set \
  --stack-name mcp-server-prod \
  --change-set-name deploy-2026-09-26 \
  --query "Changes[*].{Action:ResourceChange.Action,Type:ResourceChange.ResourceType,Id:ResourceChange.LogicalResourceId,Replacement:ResourceChange.Replacement}" \
  --output table

# Step 4 — execute (only if the preview looks safe)
aws cloudformation execute-change-set \
  --stack-name mcp-server-prod \
  --change-set-name deploy-2026-09-26

# OR — delete the change set without applying it
aws cloudformation delete-change-set \
  --stack-name mcp-server-prod \
  --change-set-name deploy-2026-09-26

Change set names must be unique within a stack but can be reused once the previous change set is executed or deleted. Convention: use a timestamp or Git SHA so the name is traceable back to the pipeline run that created it.

Reading change set output — replacement detection

The Replacement field in each change entry is the most important signal. True means the resource will be deleted and recreated. Conditional means replacement depends on the actual value — CloudFormation cannot determine statically whether it will trigger replacement. False means in-place update.

# Example describe-change-set output — JSON trimmed for readability
{
  "Changes": [
    {
      "Type": "Resource",
      "ResourceChange": {
        "Action": "Modify",
        "LogicalResourceId": "McpServerTaskDefinition",
        "ResourceType": "AWS::ECS::TaskDefinition",
        "Replacement": "True",    // WARNING: ECS task definitions are immutable — new revision created
        "Details": [
          {
            "Target": { "Attribute": "Properties", "Name": "ContainerDefinitions" },
            "Evaluation": "Static",
            "ChangeSource": "DirectModification"
          }
        ]
      }
    },
    {
      "Type": "Resource",
      "ResourceChange": {
        "Action": "Modify",
        "LogicalResourceId": "McpServerService",
        "ResourceType": "AWS::ECS::Service",
        "Replacement": "False",   // in-place update — service will roll to new task definition
        "Details": [
          {
            "Target": { "Attribute": "Properties", "Name": "TaskDefinition" },
            "Evaluation": "Dynamic",    // depends on TaskDefinition resource being replaced above
            "ChangeSource": "ResourceReference"
          }
        ]
      }
    },
    {
      "Type": "Resource",
      "ResourceChange": {
        "Action": "Remove",       // DANGER: resource will be deleted
        "LogicalResourceId": "McpServerSecurityGroup",
        "ResourceType": "AWS::EC2::SecurityGroup",
        "Replacement": "N/A"
      }
    }
  ]
}

In CI/CD, parse the JSON output and fail the pipeline if any change has Action: Remove or Replacement: True on stateful resource types (RDS, EFS, DynamoDB, S3). For stateless resources like ECS TaskDefinitions, Replacement: True is expected and safe — CloudFormation creates a new revision and the ECS service rolls over to it.

DeletionPolicy and UpdateReplacePolicy

These two policies control what happens to a resource when CloudFormation deletes or replaces it. They must be set before the destructive event, not added in the same change set that triggers deletion:

# CloudFormation YAML — protect stateful resources before any update
McpSessionsTable:
  Type: AWS::DynamoDB::Table
  DeletionPolicy: Retain          # table persists if stack is deleted
  UpdateReplacePolicy: Retain     # table persists if CF replaces it during an update
  Properties:
    TableName: mcp-sessions
    BillingMode: PAY_PER_REQUEST
    AttributeDefinitions:
      - AttributeName: sessionId
        AttributeType: S
    KeySchema:
      - AttributeName: sessionId
        KeyType: HASH

McpPostgresDatabase:
  Type: AWS::RDS::DBInstance
  DeletionPolicy: Snapshot        # creates final snapshot before deleting
  UpdateReplacePolicy: Snapshot   # creates snapshot before replacement
  Properties:
    DBInstanceClass: db.t4g.medium
    Engine: postgres
    # ...

McpEfsFileSystem:
  Type: AWS::EFS::FileSystem
  DeletionPolicy: Retain
  UpdateReplacePolicy: Retain
  Properties:
    Encrypted: true

The critical gap: DeletionPolicy and UpdateReplacePolicy only protect the resource. If you delete the entire stack without these policies set, the resources are deleted permanently. Add these policies to any stateful resource immediately after you create the stack — before the first update that could trigger replacement.

RetainExceptOnCreate (added in 2023) is useful for resources that should be retained on replacement but deleted if the stack itself is torn down during a failed creation rollback. For production MCP server databases, prefer plain Retain.

Stack Policy — protecting production resources from updates

A stack policy is a JSON document that prevents specific resources from being updated or replaced, independent of IAM permissions. It is useful as a second line of defense for production databases after DeletionPolicy/UpdateReplacePolicy are set:

{
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "Update:*",
      "Principal": "*",
      "Resource": "*"
    },
    {
      "Effect": "Deny",
      "Action": [
        "Update:Replace",
        "Update:Delete"
      ],
      "Principal": "*",
      "Resource": "LogicalResourceId/McpSessionsTable"
    },
    {
      "Effect": "Deny",
      "Action": [
        "Update:Replace",
        "Update:Delete"
      ],
      "Principal": "*",
      "Resource": "LogicalResourceId/McpPostgresDatabase"
    }
  ]
}

# Apply the stack policy to the production stack
aws cloudformation set-stack-policy \
  --stack-name mcp-server-prod \
  --stack-policy-body file://stack-policy.json

# Override for a specific change set (requires elevated IAM — audit this)
aws cloudformation execute-change-set \
  --stack-name mcp-server-prod \
  --change-set-name maintenance-window-2026-09-26 \
  --stack-policy-during-update-body '{"Statement":[{"Effect":"Allow","Action":"Update:*","Principal":"*","Resource":"*"}]}'

Stack policies cannot be removed once set (only modified). They are separate from IAM — an IAM policy that allows cloudformation:UpdateStack does not bypass a stack policy that denies Update:Replace on a specific resource.

CI/CD pattern — automated change set approval gate

A complete CI/CD pattern using change sets with automated replacement detection and manual approval for destructive changes:

# GitHub Actions workflow — change set deploy with approval gate
jobs:
  plan:
    runs-on: ubuntu-latest
    steps:
      - name: Create change set
        run: |
          aws cloudformation create-change-set \
            --stack-name mcp-server-prod \
            --change-set-name "deploy-${{ github.sha }}" \
            --template-body file://template.yaml \
            --parameters file://params-prod.json \
            --capabilities CAPABILITY_IAM

      - name: Wait for change set
        run: |
          aws cloudformation wait change-set-create-complete \
            --stack-name mcp-server-prod \
            --change-set-name "deploy-${{ github.sha }}"

      - name: Check for destructive changes
        id: check
        run: |
          CHANGES=$(aws cloudformation describe-change-set \
            --stack-name mcp-server-prod \
            --change-set-name "deploy-${{ github.sha }}" \
            --query "Changes[?ResourceChange.Action=='Remove' || (ResourceChange.Replacement=='True' && (ResourceChange.ResourceType=='AWS::RDS::DBInstance' || ResourceChange.ResourceType=='AWS::DynamoDB::Table'))]" \
            --output json)
          echo "destructive_changes=$([[ "$CHANGES" == "[]" ]] && echo false || echo true)" >> $GITHUB_OUTPUT
          echo "$CHANGES"

      - name: Require manual approval for destructive changes
        if: steps.check.outputs.destructive_changes == 'true'
        uses: trstringer/manual-approval@v1
        with:
          secret: ${{ secrets.GITHUB_TOKEN }}
          approvers: ops-team

  deploy:
    needs: plan
    runs-on: ubuntu-latest
    steps:
      - name: Execute change set
        run: |
          aws cloudformation execute-change-set \
            --stack-name mcp-server-prod \
            --change-set-name "deploy-${{ github.sha }}"

      - name: Wait for stack update
        run: |
          aws cloudformation wait stack-update-complete \
            --stack-name mcp-server-prod