AWS SSM Session Manager for MCP Servers: Zero-Bastion Interactive Access

How Session Manager works

Session Manager does not open any inbound port on your instance. The SSM Agent running on the instance establishes a persistent outbound WebSocket connection to the ssmmessages endpoint. When you call StartSession, the SSM service acts as a relay: your client connects to the SSM endpoint, the SSM service finds the matching agent connection, and a bidirectional multiplexed channel is established. All traffic is encrypted with TLS; the agent never decrypts anything except the framing headers it needs to route the channel.

This architecture means Session Manager works in three network configurations that SSH cannot handle without additional infrastructure:

• Private subnets without NAT gateway: Add VPC endpoints for ssm, ssmmessages, and ec2messages. Sessions route through the endpoint without leaving the VPC.
• Instances behind strict egress firewalls: Allow outbound HTTPS on port 443 to the SSM endpoints. No other ports needed.
• Instances with no key pair at launch: Session Manager is the only supported interactive access method — no fallback SSH needed.

Setup: CLI plugin and permissions

The instance-side prerequisites are identical to Run Command: SSM Agent installed and running, and the AmazonSSMManagedInstanceCore managed policy on the instance role. The client-side requirement is the Session Manager plugin for the AWS CLI:

# macOS
brew install --cask session-manager-plugin

# Linux (Debian/Ubuntu)
curl "https://s3.amazonaws.com/session-manager-downloads/plugin/latest/ubuntu_64bit/session-manager-plugin.deb" \
  -o /tmp/session-manager-plugin.deb
sudo dpkg -i /tmp/session-manager-plugin.deb

# Verify installation
session-manager-plugin --version

Start an interactive session:

aws ssm start-session --target i-0abc123def456789a

You land in a bash shell as the ssm-user (a user the SSM Agent creates automatically, with sudo access on Amazon Linux/Ubuntu instances). To use your own user instead, configure the session document preferences in the Systems Manager console under "Session Manager → Preferences → Linux shell profile" and set runAsEnabled: true with a runAsDefaultUser.

Port forwarding for live MCP server debugging

Port forwarding lets you connect local tools — your browser, curl, Postman, or a custom MCP client — directly to a process listening on a private instance. For MCP server debugging, this is how you verify that the server's JSON-RPC endpoint is responding correctly from the process's perspective, separate from the load balancer or service mesh in front of it.

# Forward localhost:13000 → instance:3000 (where the MCP server listens)
aws ssm start-session \
  --target i-0abc123def456789a \
  --document-name AWS-StartPortForwardingSession \
  --parameters '{"portNumber":["3000"],"localPortNumber":["13000"]}'

# In a separate terminal: probe the MCP server directly
curl -s -X POST http://localhost:13000/ \
  -H 'Content-Type: application/json' \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/list","params":{}}' | jq .

# Or forward to a Unix socket if the server uses one
aws ssm start-session \
  --target i-0abc123def456789a \
  --document-name AWS-StartPortForwardingSessionToRemoteHost \
  --parameters '{"host":["127.0.0.1"],"portNumber":["3000"],"localPortNumber":["13000"]}'

This is particularly useful when AliveMCP reports a probe failure on one specific instance but the load balancer health check still shows it as healthy — the LB may use a shallow TCP-only check, while AliveMCP's JSON-RPC tools/list probe is deeper. Port forwarding lets you reproduce exactly what AliveMCP sees from the instance's perspective.

Session logging for compliance and incident investigation

Session Manager can log all interactive session activity to CloudWatch Logs or S3. For MCP servers that handle sensitive downstream API keys, logging session activity is important both for security audits and for post-incident investigation.

Enable session logging via CDK:

import * as ssm from 'aws-cdk-lib/aws-ssm';
import * as logs from 'aws-cdk-lib/aws-logs';
import * as iam from 'aws-cdk-lib/aws-iam';

// CloudWatch Log Group for session logs
const sessionLogGroup = new logs.LogGroup(this, 'SsmSessionLogs', {
  logGroupName: '/mcp/ssm-sessions',
  retention: logs.RetentionDays.NINETY_DAYS,
  encryptionKey: kmsKey, // optional: encrypt with CMK
});

// SSM Session Manager preferences document
const sessionPreferences = new ssm.CfnDocument(this, 'SessionPreferences', {
  name: 'SSM-SessionManagerRunShell',
  documentType: 'Session',
  content: {
    schemaVersion: '1.0',
    description: 'Session Manager preferences for MCP server instances',
    sessionType: 'InteractiveCommands',
    inputs: {
      s3BucketName: '',
      s3KeyPrefix: '',
      s3EncryptionEnabled: false,
      cloudWatchLogGroupName: sessionLogGroup.logGroupName,
      cloudWatchEncryptionEnabled: true,
      cloudWatchStreamingEnabled: true,
      kmsKeyId: kmsKey.keyArn,
      runAsEnabled: false,
      runAsDefaultUser: '',
      idleSessionTimeout: '20', // minutes
      maxSessionDuration: '60',
      shellProfile: {
        linux: 'export HISTFILE=/dev/null; echo "Session on $(hostname) started at $(date) by $USER"',
      },
    },
  },
});

The HISTFILE=/dev/null in the shell profile prevents commands from being written to the instance's bash history (which could expose secrets in shared environments), while the CloudWatch streaming captures everything at the session level. The two controls serve different audiences: CloudWatch logs go to your security team; history suppression protects against secrets leaking into other users' .bash_history files.

IAM policy: restricting who can session into which instances

By default, any principal with ssm:StartSession on * can session into any SSM-managed instance in the account. For MCP server environments, scope this down: production MCP server instances should only be accessible to on-call engineers or the incident response role, not every developer.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowSessionToMcpProductionInstances",
      "Effect": "Allow",
      "Action": [
        "ssm:StartSession",
        "ssm:TerminateSession",
        "ssm:ResumeSession",
        "ssm:DescribeSessions",
        "ssm:GetConnectionStatus"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:instance/*",
        "arn:aws:ssm:*:*:document/AWS-StartSSHSession",
        "arn:aws:ssm:*:*:document/AWS-StartPortForwardingSession"
      ],
      "Condition": {
        "StringEquals": {
          "ssm:resourceTag/Service": "mcp-server",
          "ssm:resourceTag/Env": "production"
        }
      }
    },
    {
      "Sid": "DenySessionToNonMcpInstances",
      "Effect": "Deny",
      "Action": "ssm:StartSession",
      "Resource": "arn:aws:ec2:*:*:instance/*",
      "Condition": {
        "StringNotEquals": {
          "ssm:resourceTag/Service": "mcp-server"
        }
      }
    }
  ]
}

The condition key ssm:resourceTag/Service lets you scope session access by EC2 tag. Attach the Service=mcp-server tag at launch time in your CDK stack and this policy automatically covers new instances without policy updates.

VPC endpoints for private subnet instances

MCP servers in private subnets with no NAT gateway need three VPC endpoints for Session Manager to work:

import * as ec2 from 'aws-cdk-lib/aws-ec2';

// Required endpoints for SSM Session Manager in private subnets
const ssmEndpoints = [
  'com.amazonaws.us-east-1.ssm',
  'com.amazonaws.us-east-1.ssmmessages',
  'com.amazonaws.us-east-1.ec2messages',
];

for (const service of ssmEndpoints) {
  new ec2.InterfaceVpcEndpoint(this, `Endpoint-${service.split('.').pop()}`, {
    vpc,
    service: new ec2.InterfaceVpcEndpointService(service),
    subnets: { subnetType: ec2.SubnetType.PRIVATE_ISOLATED },
    securityGroups: [endpointSecurityGroup],
    privateDnsEnabled: true,
  });
}

privateDnsEnabled: true is required — the SSM Agent resolves endpoint hostnames using the standard regional DNS names (ssm.us-east-1.amazonaws.com), and the VPC endpoint's private DNS substitution makes those names resolve to the endpoint's private IP. Without this flag, the agent cannot reach the endpoint from a subnet without a route to the internet.

AliveMCP + Session Manager: the incident response workflow

AliveMCP monitors MCP servers from outside the AWS account with a real JSON-RPC probe. When a probe starts failing, the typical incident workflow is:

1. AliveMCP alert fires → on-call engineer is paged.
2. Engineer checks the AliveMCP dashboard to see which instances are failing and since when.
3. Engineer runs aws ssm start-session --target i-{id} on the failing instance — no bastion, no VPN, no SSH key lookup.
4. On the instance: check systemctl status mcp-server, journalctl -u mcp-server -n 100, and recent log files.
5. If the issue is a hung process: systemctl restart mcp-server.
6. AliveMCP probe recovers → alert resolves → post-mortem scheduled.

The entire resolution loop — from alert to fix — can happen in under five minutes when Session Manager is already configured. Without it, step 3 becomes "find the bastion, find an SSH key, configure local agent forwarding, hope the bastion's security group allows your current IP" — a 15-minute gap where the outage compounds.

Failure modes