Guide · Container Runtime
MCP Tools for Amazon ECS — task state machine, service health, and capacity providers
Amazon ECS (Elastic Container Service) is AWS's managed container orchestration service. MCP tools that manage ECS workloads — deploying new task definitions, checking service health, running one-off tasks, or diagnosing deployment failures — must understand three ECS-specific behaviors that differ from local container runtimes: the task state machine (eight states between task launch request and task termination, with different health meanings at each), the service health vs task health distinction (a service with runningCount === desiredCount is not necessarily serving traffic — the tasks may be in health check grace period, or the load balancer target group may have deregistered them), and capacity provider availability (Fargate and EC2 capacity providers can be unavailable due to regional capacity constraints, which causes tasks to get stuck in PROVISIONING indefinitely without a clear error).
TL;DR
Use the AWS SDK v3 @aws-sdk/client-ecs. For task health: poll DescribeTasks until lastStatus reaches a terminal state (RUNNING, STOPPED) — PROVISIONING and PENDING are not terminal. For service health: check DescribeServices for runningCount === desiredCount AND inspect the deployments array — a service with runningCount === desiredCount but with an active deployment still in progress is mid-rollout, not stable. For capacity issues: check DescribeCapacityProviders for capacityProviderStatus and watch for tasks stuck in PROVISIONING beyond 5 minutes. Register your ECS service's load balancer URL with AliveMCP for external monitoring.
Connecting to ECS: AWS SDK v3 and IAM authentication
ECS API calls are authenticated via AWS IAM. For MCP servers running on AWS infrastructure (EC2, Lambda, ECS itself), use the default credential chain — the SDK automatically picks up the instance metadata service (IMDS) IAM role credentials. For MCP servers running outside AWS, use environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION) or a shared credentials file.
import { ECSClient, DescribeTasksCommand, DescribeServicesCommand,
RunTaskCommand, ListTasksCommand, DescribeCapacityProvidersCommand } from '@aws-sdk/client-ecs';
const ecs = new ECSClient({ region: process.env.AWS_REGION || 'us-east-1' });
// Minimum IAM permissions for read-only MCP tools:
// ecs:DescribeTasks, ecs:DescribeServices, ecs:ListTasks,
// ecs:DescribeTaskDefinition, ecs:ListTaskDefinitions,
// ecs:DescribeClusters, ecs:DescribeCapacityProviders
// For write tools (run task, update service):
// ecs:RunTask, ecs:StopTask, ecs:UpdateService, ecs:RegisterTaskDefinition
// Also needed for VPC networking: ec2:DescribeSubnets, ec2:DescribeSecurityGroups
// For ECS exec (container shell): ssmmessages:CreateControlChannel and related SSM permissions
The ECS API is regional. An MCP tool that manages ECS across multiple regions must instantiate separate ECSClient instances per region. Always surface the region in tool responses — a task ARN that looks healthy in us-east-1 may be completely separate from the service in eu-west-1 that the agent is asking about.
Task state machine: eight states, only two are terminal
An ECS task moves through up to eight distinct states between launch and termination. Most are transient — they represent work in progress and require polling, not a final result. Only RUNNING and STOPPED are stable states.
| State | Meaning | Typical duration |
|---|---|---|
PROVISIONING | ECS is acquiring capacity (ENI attachment for Fargate, finding a container instance for EC2) | 10–60s; over 5m = problem |
PENDING | Capacity acquired; pulling container images | 10s–5m depending on image size |
ACTIVATING | Registering with load balancer target group (if service-connected) | 5–30s |
RUNNING | All containers have started; may still be in health check grace period | Stable (until stopped) |
DEACTIVATING | Deregistering from load balancer target group | 30–60s (LB connection draining) |
STOPPING | SIGTERM sent to containers; waiting for graceful shutdown | Up to stopTimeout seconds |
DEPROVISIONING | Releasing ENI, releasing capacity reservation | 10–30s |
STOPPED | Task terminated; stopCode and stoppedReason explain why | Permanent |
// Poll task until it reaches a stable state
async function waitForTask(cluster, taskArn, timeoutMs = 300_000) {
const TERMINAL = new Set(['RUNNING', 'STOPPED']);
const start = Date.now();
while (Date.now() - start < timeoutMs) {
const { tasks } = await ecs.send(new DescribeTasksCommand({
cluster,
tasks: [taskArn]
}));
const task = tasks?.[0];
if (!task) throw new Error('Task not found');
const state = task.lastStatus;
if (TERMINAL.has(state)) {
return {
state,
stopCode: task.stopCode, // e.g. 'EssentialContainerExited'
stoppedReason: task.stoppedReason, // human-readable explanation
containers: task.containers?.map(c => ({
name: c.name,
lastStatus: c.lastStatus,
exitCode: c.exitCode,
reason: c.reason,
})),
};
}
// Check for stuck PROVISIONING (capacity problem)
if (state === 'PROVISIONING') {
const elapsedMin = (Date.now() - start) / 60_000;
if (elapsedMin > 5) {
return { state: 'STUCK_PROVISIONING', elapsed_minutes: elapsedMin };
}
}
await new Promise(r => setTimeout(r, 5000)); // poll every 5s
}
throw new Error('Timeout waiting for task state');
}
When a task stops, always check stopCode and stoppedReason. The most important stop codes for MCP tool diagnostics: EssentialContainerExited (the container defined with essential: true in the task definition exited — check the container's exitCode and reason fields), TaskFailedToStart (usually an image pull failure or resource constraint), CannotPullContainerError (image not accessible — check ECR permissions, private registry credentials, or network access to the registry), and OutOfMemoryError (the task's containers exceeded the task memory limit — increase task memory in the task definition).
Service health vs task health: running count is not enough
An ECS service maintains a desired number of running tasks. The most obvious health check — comparing runningCount to desiredCount — is necessary but not sufficient for determining whether a service is healthy and serving traffic.
Three gaps between "running == desired" and "actually healthy":
1. Health check grace period. When a new task starts in a service connected to a load balancer, ECS waits for the healthCheckGracePeriodSeconds before starting health checks via the target group. During this period, the task counts as running but is not yet in the load balancer's healthy rotation. A service can report runningCount === desiredCount while all its tasks are still in grace period. Check the service's healthCheckGracePeriodSeconds setting and compare against the task's launch time.
2. Active deployments. The DescribeServices response includes a deployments array. When a deployment is in progress, there are two entries: the new deployment (status: 'PRIMARY') and the old deployment (status: 'ACTIVE'). A service with runningCount === desiredCount but with both PRIMARY and ACTIVE deployments present is mid-rollout — some tasks are running the old version, some the new version. This is normal during a deployment but should not be reported as "stable healthy."
3. Load balancer target group health. ECS service health (via DescribeServices) reflects the ECS scheduler's view. Load balancer target group health (via Elastic Load Balancing APIs) reflects whether the targets are actually passing health checks. A task can be ECS-RUNNING but ALB-unhealthy (the application inside is returning 5xx, or the health check endpoint is timing out).
async function ecsServiceHealth(cluster, serviceName) {
const { services } = await ecs.send(new DescribeServicesCommand({
cluster,
services: [serviceName],
}));
const svc = services?.[0];
if (!svc) return { healthy: false, reason: 'service_not_found' };
const primaryDeployment = svc.deployments?.find(d => d.status === 'PRIMARY');
const oldDeployments = svc.deployments?.filter(d => d.status === 'ACTIVE');
return {
status: svc.status, // 'ACTIVE' means the service definition is active
runningCount: svc.runningCount,
desiredCount: svc.desiredCount,
pendingCount: svc.pendingCount,
deploymentInProgress: oldDeployments?.length > 0,
primaryDeploymentStatus: primaryDeployment?.rolloutState, // 'COMPLETED', 'IN_PROGRESS', 'FAILED'
events: svc.events?.slice(0, 3).map(e => ({ // last 3 service events
message: e.message,
createdAt: e.createdAt,
})),
// Running == desired AND no in-progress deployment AND rollout completed
stable: svc.runningCount === svc.desiredCount
&& svc.pendingCount === 0
&& oldDeployments?.length === 0
&& primaryDeployment?.rolloutState === 'COMPLETED',
};
}
Capacity providers: Fargate availability and EC2 ASG scaling
ECS tasks run on capacity provided by either AWS Fargate or EC2 container instances. When capacity is unavailable, tasks get stuck in PROVISIONING state without a clear error in the ECS API — the failure manifests only as a service event ("service was unable to place a task") or a task stuck indefinitely in PROVISIONING.
Fargate capacity constraints: Fargate capacity is regional and can be constrained during high-demand periods. Fargate Spot capacity (for FARGATE_SPOT capacity provider) is more likely to be unavailable or interrupted. If tasks are stuck in PROVISIONING, check whether there are recent Fargate capacity constraint events by looking at service events in DescribeServices. Use Fargate capacity providers with a FARGATE base and FARGATE_SPOT weight to fall back to on-demand when Spot is unavailable.
EC2 ASG capacity providers: ECS cluster Auto Scaling Groups manage EC2 instance pools. Capacity is constrained when the ASG has reached its maximum size but demand exceeds supply. Check the capacity provider with DescribeCapacityProviders — the capacityProviderStatus field shows ACTIVE or INACTIVE. For ASG capacity providers, check the ASG's current and maximum size via the EC2 Auto Scaling API to determine if the cluster can accept more tasks.
async function capacityProviderHealth(providerNames) {
const { capacityProviders } = await ecs.send(
new DescribeCapacityProvidersCommand({ capacityProviders: providerNames })
);
return capacityProviders?.map(cp => ({
name: cp.name,
status: cp.status, // ACTIVE, INACTIVE, DELETING
// For Fargate/Fargate Spot: no ASG details
// For EC2 ASG capacity providers:
asg: cp.autoScalingGroupProvider ? {
autoScalingGroupArn: cp.autoScalingGroupProvider.autoScalingGroupArn,
managedScaling: cp.autoScalingGroupProvider.managedScaling?.status,
targetCapacity: cp.autoScalingGroupProvider.managedScaling?.targetCapacity,
} : null,
}));
}
ECS Exec: running commands inside Fargate tasks
ECS Exec (analogous to docker exec) lets you run commands inside running Fargate or EC2 tasks via AWS Systems Manager Session Manager. It requires the task's IAM execution role to have SSM permissions and the enableExecuteCommand: true flag on the service or task.
import { ExecuteCommandCommand } from '@aws-sdk/client-ecs';
async function execInEcsTask(cluster, taskArn, containerName, command) {
// Requires: task was launched with enableExecuteCommand: true
// Requires: task execution role has ssmmessages:* and ssm:* permissions
const { session } = await ecs.send(new ExecuteCommandCommand({
cluster,
task: taskArn,
container: containerName,
interactive: true,
command, // e.g., '/bin/sh -c "curl -sf http://localhost:8080/health"'
}));
// session.streamUrl and session.tokenValue are passed to the SSM SDK
// to establish a WebSocket connection for interactive I/O
return session;
}
// Check if ECS Exec is available for a task
async function ecsExecAvailable(cluster, taskArn) {
const { tasks } = await ecs.send(new DescribeTasksCommand({ cluster, tasks: [taskArn] }));
const task = tasks?.[0];
return task?.enableExecuteCommand === true && task?.lastStatus === 'RUNNING';
}
ECS Exec is useful for MCP tools that need to run one-off diagnostic commands inside Fargate containers. The session is established via SSM WebSockets — it works even when the container has no inbound network access, as long as the ECS task can reach the SSM endpoint (via VPC endpoint or NAT gateway). For health check commands that only need to retrieve output (not interactive), prefer running a separate short-lived ECS task with a specific command over ECS Exec — short-lived tasks are tracked in CloudWatch and have a clean audit trail.
Frequently asked questions
How do I run a one-off ECS task from an MCP tool and get its output?
Use RunTaskCommand to start the task, poll DescribeTasksCommand until the task reaches STOPPED state, then read the container's logs from CloudWatch Logs. The steps: (1) Call RunTask with your task definition ARN, cluster, network configuration (subnets and security groups for Fargate tasks), and a specific command override if needed. (2) Extract the task ARN from the response. (3) Poll DescribeTasks every 5-10 seconds until lastStatus is STOPPED. (4) Check the container's exitCode — 0 means success. (5) Fetch logs from CloudWatch Logs using the log group and log stream defined in the task definition's logConfiguration. The log stream name for ECS tasks follows the pattern: {logStreamPrefix}/{containerName}/{taskId} where taskId is the last part of the task ARN. Use @aws-sdk/client-cloudwatch-logs with GetLogEventsCommand to retrieve the output. Note: logs may not be immediately available when the task enters STOPPED state — add a short delay (5-10s) before fetching logs.
What is the difference between ECS task definitions and tasks?
A task definition is an immutable template that describes how to run containers: the Docker image, CPU and memory limits, port mappings, environment variables, IAM roles, log configuration, and volume mounts. Task definitions are versioned — every change creates a new revision with an incremented number (e.g., my-app:42). A task is a running instance created from a task definition — it is the actual container workload executing on a Fargate instance or EC2 container instance. The relationship is like a class vs an instance: the task definition is the blueprint, the task is the execution. ECS services maintain a target number of running tasks from a specified task definition revision; updating the service to a newer revision triggers ECS to replace old tasks with new ones following the configured deployment strategy. For MCP tools, always capture the full task definition ARN (including revision number) in tool responses so agents can trace which code version a running task is executing.
How does ECS service auto-scaling work and what should my MCP tool check?
ECS services integrate with Application Auto Scaling to adjust desiredCount based on CloudWatch metrics (CPU utilization, memory utilization, ALB request count per target, custom metrics). An MCP tool monitoring service health should check whether auto-scaling is active (reducing confusion between intentional scale-down and service degradation). Use @aws-sdk/client-application-auto-scaling with DescribeScalableTargetsCommand (filtering by serviceNamespace: 'ecs' and resourceId: 'service/{cluster}/{serviceName}') to check whether auto-scaling is configured. If auto-scaling is active, a service's desiredCount can be lower than expected during off-peak hours — this is expected behavior, not a failure. Surface the auto-scaling min/max capacity alongside the current running count in health responses so agents can distinguish "scaled down intentionally" from "crashed to zero".
How do I distinguish a task that was stopped by ECS from one that crashed?
Check the task's stopCode field in DescribeTasks response. EssentialContainerExited means the container's process exited — the exitCode (0 = clean exit, non-zero = error) and reason on the specific container tell you whether it was intentional. TaskFailedToStart means ECS couldn't even start the task (image pull failure, resource constraint). UserInitiated means a human or automation called StopTask explicitly. ServiceSchedulerInitiated means ECS stopped the task as part of a deployment or scale-in event — this is expected and not a failure. SpotInterruption means the Fargate Spot instance was reclaimed by AWS — relevant only for Fargate Spot capacity. For MCP tools diagnosing unexplained task stops, filter for tasks with stopCode !== 'UserInitiated' and stopCode !== 'ServiceSchedulerInitiated' to surface only unintentional terminations. Always include stoppedReason in the tool response — it is a human-readable explanation written by ECS that often includes the specific error message.
How do I monitor ECS services with AliveMCP?
ECS services typically sit behind an Application Load Balancer or a Network Load Balancer. Register the load balancer's DNS name and the service's health check path with AliveMCP. For example, if your ECS service runs an MCP server at https://mcp-alb.example.com/mcp, register that URL. AliveMCP will probe it every 60 seconds — if the probe fails, all tasks behind the ALB are either unhealthy or the ALB itself has an issue. For the AliveMCP probe to work: (1) The ALB must be internet-accessible or AliveMCP must be on the same VPC; (2) The ECS service's target group health check path (/health or similar) must return 200 when the application is healthy; (3) The security group on the ALB must allow inbound HTTPS from AliveMCP's probe IPs. The combination of AliveMCP's external HTTP probe (which detects application-level failures) and ECS service events (which detect infrastructure-level failures) gives full coverage. Register both the ALB URL with AliveMCP and an ECS CloudWatch alarm on HealthyHostCount for the target group for belt-and-suspenders monitoring.
Further reading
- MCP Tools for Docker Engine API — container lifecycle, exec vs attach, image cache
- MCP Tools for containerd — gRPC API, task lifecycle, snapshot management
- MCP Tools for Podman REST API — rootless containers, pods, and quadlets
- MCP Tools for Google Cloud Run — revision lifecycle, traffic splitting, and cold start
- MCP Servers on Kubernetes — readiness probes, PDBs, and session affinity
- MCP Server Health Checks — implementing and monitoring protocol probes
- MCP Server Zero-Downtime Deployment — rolling updates and blue/green strategies
- MCP Tools for DevOps — patterns across container runtimes and orchestrators