AWS CloudWatch Synthetics · 2026-11-06 · Synthetics arc
AWS CloudWatch Synthetics for MCP Servers: Canary Creation, Script Writing, Run Monitoring, and Failure Alarms
AliveMCP probes every registered MCP endpoint every 60 seconds — and the CloudWatch Synthetics integration failures we see cluster around four mistakes that nearly every MCP server team makes in their first week: a canary creation that calls CreateCanaryCommand and then immediately queries for run results, not realizing that CreateCanaryCommand registers the canary definition but does not start it — the canary sits in READY state until you call StartCanaryCommand separately, and GetCanaryRunsCommand returns an empty list until the first scheduled run fires; a canary script that uses ES module import syntax and either fails to upload or fails at runtime, because the Synthetics Lambda environment requires CommonJS (require) — the built-in Synthetics and SyntheticsLogger modules are only accessible via require('Synthetics'), not import, and there is no workaround for this restriction; a canary that enters ERROR state immediately after creation because the team used the same IAM role for both the MCP server and the canary execution, not realizing that Synthetics needs two separate roles — the MCP server's role calls the Synthetics control-plane API and needs synthetics:CreateCanary plus iam:PassRole with a iam:PassedToService: "lambda.amazonaws.com" condition, while the canary execution role needs a lambda.amazonaws.com trust policy and permissions to write artifacts to S3; and an alarm on SuccessPercent that stays in INSUFFICIENT_DATA state for days because the team set Period: 60 but their canary runs every 5 minutes, so CloudWatch expects one datapoint every 60 seconds and the actual datapoints arrive every 300 seconds — Period must equal the canary's schedule interval exactly so each evaluation period covers exactly one canary run. Four CloudWatch Synthetics patterns form the complete synthetic monitoring stack for MCP servers. Canary creation and management with CreateCanaryCommand and StartCanaryCommand is the foundation: you define the monitoring configuration, attach the execution role, point it at an artifact bucket, and start the schedule — the canary then runs on its own without any further intervention from your MCP server. Canary scripts are Node.js files that run inside a managed Lambda environment and use the built-in Synthetics module to define named steps — each step is a discrete check that appears individually in the CloudWatch console and contributes to the per-run pass/fail result. Reading run results with GetCanaryRunsCommand and DescribeCanariesLastRunCommand lets MCP server tools surface current uptime status, compute pass rates, detect outage windows from run history, and retrieve the S3 artifact logs for failed runs. Failure alarms with PutMetricAlarmCommand on the SuccessPercent metric close the loop — when canary runs start failing, CloudWatch fires the alarm and publishes to an SNS topic, which routes to email, Slack, or PagerDuty without any polling from your server. This guide synthesizes all four Synthetics patterns into a complete synthetic monitoring integration stack for MCP server builders.
TL;DR
- Canary creation — install
@aws-sdk/client-synthetics; constructSyntheticsClientonce; callCreateCanaryCommandwithName(1–21 chars, lowercase letters/numbers/hyphens only),Code(S3 bucket+key+handler or inline zip),ArtifactS3Location(S3 URI prefix for run artifacts),ExecutionRoleArn(a separate IAM role withlambda.amazonaws.comtrust),RuntimeVersion: 'syn-nodejs-puppeteer-9.1', andSchedule: { Expression: 'rate(5 minutes)' }; then callStartCanaryCommand— CreateCanary alone does NOT start runs: The MCP server role needssynthetics:CreateCanary,synthetics:StartCanary, andiam:PassRolescoped to the execution role ARN with the condition"iam:PassedToService": "lambda.amazonaws.com". Without this condition, Synthetics can receive the creation request but fails to provision the underlying Lambda, causing the canary to transition toERRORstate. The execution role is a separate IAM role that Synthetics assumes when running the canary Lambda — it needss3:PutObjecton the artifact bucket,logs:CreateLogStreamandlogs:PutLogEventson/aws/lambda/cwsyn-*log groups, andcloudwatch:PutMetricData(Synthetics auto-publishes run metrics). The canary lifecycle is:CREATING → READYafterCreateCanaryCommand, thenSTARTING → RUNNINGafterStartCanaryCommand. TheRUNNINGstate means the canary is scheduled and active — it does not mean a run is currently in-flight. Individual run results are separate. - Canary script writing — the script must use CommonJS (
require), not ESM (import); the built-in modules arerequire('Synthetics')andrequire('SyntheticsLogger'); usesynthetics.executeStep('stepName', async () => { /* throw on failure */ })for named steps orsynthetics.executeHttpStepfor simple 2xx checks; package the script as a zip with the file atnodejs/node_modules/<filename>.js— placing it at the zip root causes module-not-found at runtime; setHandler: 'filename.handler'inCreateCanaryCommand.Code: TheHandlerfield resolves relative tonodejs/node_modules/— so"api-check.handler"maps tonodejs/node_modules/api-check.jsexportinghandler. Thenodejs/node_modules/path prefix is required by the Synthetics runtime; the most common packaging mistake is putting the script at the zip root or atnodejs/api-check.js(without thenode_modules/segment). npm dependencies must be bundled in the zip — the canary Lambda has no outbound internet access to install packages at runtime. Use thearchivernpm package to generate zips programmatically from your MCP server: append the script content witharchive.append(content, { name: 'nodejs/node_modules/api-check.js' }), then upload the resulting buffer to S3 withPutObjectCommand, then callCreateCanaryCommandorUpdateCanaryCommandreferencing the new S3 key. - Reading run results — call
GetCanaryRunsCommand({ Name: canaryName, MaxResults: 10 })for recent run history; eachCanaryRunhasStatus.State(PASSED,FAILED, orRUNNING),Timeline.Started/Timeline.Completed, andArtifactS3Location; for a dashboard showing the latest run across all canaries, useDescribeCanariesLastRunCommandinstead of callingGetCanaryRunsCommandper canary; paginate both commands withNextToken: Run data is retained forSuccessRetentionPeriodInDays(for passed runs) andFailureRetentionPeriodInDays(for failed runs) — both set at canary creation time, defaulting to 31 days. The runStatus.Stateis completely separate from the canary lifecycleStatus.State: a canary can be inRUNNINGlifecycle state (active and scheduled) while its most recent run result isFAILED. TheArtifactS3Locationis an S3 URI prefix — parse out the bucket and key prefix with a string split and useListObjectsV2Commandto enumerate files, thenGetObjectCommandto downloadcanary-run.logfor failure diagnosis. For outage detection: iterate run history oldest-first, track the firstFAILEDrun asoutageStart, and the nextPASSEDrun asoutageEnd. - Failure alarms — use
PutMetricAlarmCommandfrom@aws-sdk/client-cloudwatchwithNamespace: 'CloudWatchSynthetics',MetricName: 'SuccessPercent',Dimensions: [{ Name: 'CanaryName', Value: canaryName }]; setPeriodto match the canary schedule interval exactly (e.g.300for a 5-minute canary); useEvaluationPeriods: 2andDatapointsToAlarm: 2to alarm after 2 consecutive failures; wire to an SNS topic viaAlarmActions: [snsTopicArn]for email, Slack, or PagerDuty delivery: CloudWatch Synthetics auto-publishes metrics to theCloudWatchSyntheticsnamespace after every run — you do not emit them manually. TheSuccessPercentmetric is 100 for aPASSEDrun and 0 for aFAILEDrun; it is not 50 for a partially-passing multi-step run — a single step failure marks the entire run asFAILEDand emitsSuccessPercent: 0. SettingDatapointsToAlarm: 2out ofEvaluationPeriods: 2means the alarm fires only after two consecutive failures, which eliminates false positives from single transient network blips. For higher-stakes endpoints, set both to 1 to alarm on the first failure. TheTreatMissingData: 'notBreaching'setting means that if the canary is stopped or hasn't run yet, the alarm staysOKrather than firing — use'breaching'for SLA-critical endpoints where a stopped canary should itself be an alert.
Pattern 1: CreateCanaryCommand, canary lifecycle, and dual IAM roles
When an MCP server tool needs to provision synthetic monitoring — spinning up a scheduled health check for a new API endpoint, rotating canary scripts, or programmatically managing a fleet of monitors — CloudWatch Synthetics canary creation is the right starting point. The model is: define the canary once with CreateCanaryCommand, then start it with StartCanaryCommand. After that, Synthetics manages the schedule without any further API calls from your server.
Installing the SDK and creating the client
CloudWatch Synthetics uses a dedicated package separate from @aws-sdk/client-cloudwatch. Install both if you need to both manage canaries and read their metrics or create alarms.
npm install @aws-sdk/client-synthetics @aws-sdk/client-cloudwatch
Construct the client once at module initialization — it is stateless and safe to share across concurrent tool calls.
import { SyntheticsClient } from '@aws-sdk/client-synthetics';
const synthetics = new SyntheticsClient({
region: process.env.AWS_REGION ?? 'us-east-1',
// Credentials resolve from the standard chain:
// 1. AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY env vars
// 2. ~/.aws/credentials
// 3. ECS task role / Lambda execution role / EC2 instance profile (preferred)
});
Creating and starting a canary
The critical thing to get right: CreateCanaryCommand registers the canary definition and transitions it to READY state. It does not start running. You must call StartCanaryCommand separately to begin the schedule.
import {
SyntheticsClient,
CreateCanaryCommand,
StartCanaryCommand,
type CreateCanaryCommandInput,
} from '@aws-sdk/client-synthetics';
const synthetics = new SyntheticsClient({ region: process.env.AWS_REGION ?? 'us-east-1' });
export async function createAndStartCanary(params: {
name: string; // 1–21 chars, lowercase letters, numbers, hyphens only
scriptS3Bucket: string; // S3 bucket containing the canary script zip
scriptS3Key: string; // S3 key of the zip file, e.g. 'canaries/api-check.zip'
handler: string; // 'api-check.handler' — filename (no .js) + export name
executionRoleArn: string; // separate IAM role that Synthetics assumes to run the Lambda
artifactBucket: string; // S3 bucket where run artifacts (logs, screenshots) are stored
intervalMinutes?: number; // schedule interval — defaults to 5
}): Promise<string> { // returns canary ARN
const expression = `rate(${params.intervalMinutes ?? 5} minutes)`;
const input: CreateCanaryCommandInput = {
Name: params.name,
Code: {
S3Bucket: params.scriptS3Bucket,
S3Key: params.scriptS3Key,
Handler: params.handler, // e.g. 'api-check.handler'
},
// S3 URI prefix where Synthetics writes screenshots, logs, and HAR files
ArtifactS3Location: `s3://${params.artifactBucket}/canary-artifacts/${params.name}/`,
// IAM role that the canary Lambda execution assumes — NOT your MCP server's role
ExecutionRoleArn: params.executionRoleArn,
// Current recommended runtime: Node.js 18 + Puppeteer 21
RuntimeVersion: 'syn-nodejs-puppeteer-9.1',
Schedule: {
Expression: expression, // 'rate(5 minutes)', 'rate(1 hour)', 'cron(0 * * * ? *)'
},
RunConfig: {
TimeoutInSeconds: 60,
MemoryInMB: 960, // default; increase if script uses Puppeteer browser
ActiveTracing: false, // set true to emit X-Ray traces
},
SuccessRetentionPeriodInDays: 14,
FailureRetentionPeriodInDays: 30,
Tags: { 'created-by': 'mcp-tool' },
};
const createResp = await synthetics.send(new CreateCanaryCommand(input));
const arn = createResp.Canary?.Arn ?? '';
// CreateCanaryCommand leaves the canary in READY state.
// Call StartCanaryCommand to begin the scheduled runs.
await synthetics.send(new StartCanaryCommand({ Name: params.name }));
return arn;
}
Canary lifecycle state machine
The Status.State field on the canary object represents the provisioning lifecycle, not whether recent runs are passing or failing. Understanding the state machine prevents incorrect assumptions about canary health.
| State | Meaning | Transitions to |
|---|---|---|
CREATING | CreateCanaryCommand accepted, resources provisioning | READY or ERROR |
READY | Created but not running — waiting for StartCanaryCommand | STARTING (after Start) |
STARTING | Start accepted, scheduler enabling | RUNNING |
RUNNING | Canary is active and executing on schedule | STOPPING, UPDATING |
UPDATING | UpdateCanaryCommand in progress — runs paused | RUNNING (after update completes) |
STOPPING | StopCanaryCommand accepted, in-flight run completing | STOPPED |
STOPPED | Halted — no new runs until StartCanaryCommand called again | STARTING (after Start) |
ERROR | Provisioning or configuration error — check StateReason | Can retry Start after fixing underlying issue |
DELETED | DeleteCanaryCommand completed | Terminal |
The dual IAM role requirement
The most common canary creation failure — the canary creates successfully but enters ERROR state within seconds — is caused by getting the IAM roles wrong. CloudWatch Synthetics requires two distinct IAM principals with different trust policies and permissions.
// Policy for the MCP server role (the role that calls the Synthetics API)
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "SyntheticsControlPlane",
"Effect": "Allow",
"Action": [
"synthetics:CreateCanary",
"synthetics:UpdateCanary",
"synthetics:DeleteCanary",
"synthetics:StartCanary",
"synthetics:StopCanary",
"synthetics:GetCanary",
"synthetics:ListCanaries",
"synthetics:GetCanaryRuns",
"synthetics:DescribeCanaries",
"synthetics:DescribeCanariesLastRun"
],
"Resource": "*"
},
{
"Sid": "PassExecutionRole",
"Effect": "Allow",
"Action": "iam:PassRole",
"Resource": "arn:aws:iam::123456789012:role/canary-execution-role",
// This condition is required — Synthetics uses it to create a Lambda
// backed by the execution role. Without it, canary enters ERROR state.
"Condition": {
"StringEquals": { "iam:PassedToService": "lambda.amazonaws.com" }
}
},
{
"Sid": "UploadCanaryScripts",
"Effect": "Allow",
"Action": ["s3:PutObject", "s3:GetObject"],
"Resource": "arn:aws:s3:::my-canary-scripts/*"
}
]
}
// Trust policy for the canary execution role
// (the role that Synthetics assumes to run the canary Lambda function)
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Principal": { "Service": "lambda.amazonaws.com" },
"Action": "sts:AssumeRole"
}]
}
// Permissions for the canary execution role
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": ["s3:PutObject"],
"Resource": "arn:aws:s3:::my-canary-artifacts/*"
},
{
"Effect": "Allow",
"Action": ["s3:GetObject"],
"Resource": "arn:aws:s3:::my-canary-scripts/*"
},
{
"Effect": "Allow",
"Action": ["logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents"],
"Resource": "arn:aws:logs:*:*:log-group:/aws/lambda/cwsyn-*"
},
{
"Effect": "Allow",
"Action": ["cloudwatch:PutMetricData"],
"Resource": "*",
"Condition": { "StringEquals": { "cloudwatch:namespace": "CloudWatchSynthetics" } }
}
]
}
When the canary enters ERROR state immediately after creation, check Status.StateReason from GetCanaryCommand — it will tell you which permission is missing. The two most common causes are: the execution role's trust policy does not list lambda.amazonaws.com as a trusted service, and the execution role does not have s3:PutObject on the artifact bucket (Synthetics tries to write artifacts on the first run and fails).
Pattern 2: Writing CloudWatch Synthetics canary scripts
The canary script is a Node.js file that runs inside a managed Lambda environment provisioned by Synthetics. The Lambda environment is not your MCP server's Node.js process — it is a separate execution context in AWS with different module availability, different network access, and a strict CommonJS requirement.
Script structure and the CommonJS requirement
All canary scripts must use CommonJS (require). The Synthetics runtime does not support ES module syntax (import / export). Even if your MCP server project uses ESM throughout, the canary scripts must be written as CommonJS files.
// api-check.js — CloudWatch Synthetics canary script
// Must use CommonJS (require), not ESM (import).
// 'Synthetics' and 'SyntheticsLogger' are built-in — do not npm install them.
const synthetics = require('Synthetics');
const log = require('SyntheticsLogger');
const https = require('https');
exports.handler = async () => {
// executeStep creates a named step in the CloudWatch console.
// Each step is timed individually. If the step function throws,
// the step is marked FAILED and the entire run is marked FAILED.
await synthetics.executeStep('check-health-endpoint', async () => {
const response = await httpsGet('https://api.example.com/health');
log.info('Health response', { status: response.statusCode, body: response.body });
if (response.statusCode !== 200) {
throw new Error(`Expected 200, got ${response.statusCode}`);
}
const json = JSON.parse(response.body);
if (json.status !== 'ok') {
throw new Error(`Expected status:'ok', got: ${json.status}`);
}
});
await synthetics.executeStep('check-api-latency', async () => {
const start = Date.now();
const response = await httpsGet('https://api.example.com/v1/ping');
const latencyMs = Date.now() - start;
if (response.statusCode >= 500) {
throw new Error(`Server error: ${response.statusCode}`);
}
// Fail the step if latency exceeds 2 seconds
if (latencyMs > 2000) {
throw new Error(`Latency too high: ${latencyMs}ms`);
}
});
};
function httpsGet(url) {
return new Promise((resolve, reject) => {
const req = https.get(url, (res) => {
let body = '';
res.on('data', chunk => { body += chunk; });
res.on('end', () => resolve({ statusCode: res.statusCode, body }));
});
req.on('error', reject);
req.setTimeout(10000, () => {
req.destroy();
reject(new Error('Request timeout after 10s'));
});
});
}
Using executeHttpStep for simpler checks
For basic API health checks that only need to verify a 2xx response code, synthetics.executeHttpStep handles the request and assertion in one call, with built-in credential redaction for the artifact logs.
const synthetics = require('Synthetics');
exports.handler = async () => {
const stepConfig = {
includeRequestHeaders: true,
includeResponseHeaders: true,
includeResponseBody: true,
// Headers listed here are redacted in run artifacts — use for auth headers
restrictedHeaders: ['Authorization', 'x-api-key'],
continueOnStepFailure: false,
};
await synthetics.executeHttpStep(
'verify-api-health',
{
hostname: 'api.example.com',
method: 'GET',
path: '/health',
port: 443,
protocol: 'https:',
headers: {
'Accept': 'application/json',
'x-api-key': process.env.API_KEY ?? '',
},
},
null, // no request body for GET
stepConfig,
);
};
Zip packaging: the nodejs/node_modules/ path requirement
The Synthetics runtime resolves the Handler field relative to nodejs/node_modules/ inside the zip. This means the canary script must be at nodejs/node_modules/api-check.js in the zip — not at api-check.js at the zip root, and not at nodejs/api-check.js. Getting this wrong produces a module-not-found error at canary run time, which is not visible until after the first run attempt.
# Build the canary package (shell commands for reference)
mkdir -p canary-package/nodejs/node_modules
# Copy the script into the required path
cp api-check.js canary-package/nodejs/node_modules/
# Install dependencies into the nodejs directory (if any)
cd canary-package/nodejs
npm install --prefix . some-dependency
# Create zip — the nodejs/ directory must be at the root of the zip
cd ..
zip -r ../api-check.zip nodejs/
# In CreateCanaryCommand:
# Code: { S3Bucket: 'my-scripts', S3Key: 'api-check.zip', Handler: 'api-check.handler' }
Programmatic zip upload from an MCP tool
To deploy a canary script from an MCP server tool — without a CI/CD step — generate the zip in memory using the archiver package, upload it to S3, and reference it in CreateCanaryCommand or UpdateCanaryCommand.
import { S3Client, PutObjectCommand } from '@aws-sdk/client-s3';
import { SyntheticsClient, CreateCanaryCommand } from '@aws-sdk/client-synthetics';
import archiver from 'archiver';
const s3 = new S3Client({ region: process.env.AWS_REGION ?? 'us-east-1' });
const synthetics = new SyntheticsClient({ region: process.env.AWS_REGION ?? 'us-east-1' });
export async function deployCanary(params: {
canaryName: string;
scriptContent: string; // CommonJS source code of the canary script
scriptFilename: string; // e.g. 'api-check.js'
targetUrl: string;
executionRoleArn: string;
artifactBucket: string;
scriptBucket: string;
}): Promise<string> {
// 1. Build zip in memory with archiver
const zipBuffer = await zipCanaryScript(params.scriptContent, params.scriptFilename);
// 2. Upload to S3 with a versioned key
const s3Key = `canaries/${params.canaryName}/${Date.now()}.zip`;
await s3.send(new PutObjectCommand({
Bucket: params.scriptBucket,
Key: s3Key,
Body: zipBuffer,
ContentType: 'application/zip',
}));
// 3. Create canary referencing the uploaded script
const response = await synthetics.send(new CreateCanaryCommand({
Name: params.canaryName,
Code: {
S3Bucket: params.scriptBucket,
S3Key: s3Key,
Handler: `${params.scriptFilename.replace('.js', '')}.handler`,
},
ArtifactS3Location: `s3://${params.artifactBucket}/canary-artifacts/${params.canaryName}/`,
ExecutionRoleArn: params.executionRoleArn,
RuntimeVersion: 'syn-nodejs-puppeteer-9.1',
Schedule: { Expression: 'rate(5 minutes)' },
RunConfig: {
TimeoutInSeconds: 60,
EnvironmentVariables: { TARGET_URL: params.targetUrl },
},
SuccessRetentionPeriodInDays: 14,
FailureRetentionPeriodInDays: 30,
}));
return response.Canary?.Arn ?? '';
}
async function zipCanaryScript(content: string, filename: string): Promise<Buffer> {
return new Promise((resolve, reject) => {
const chunks: Buffer[] = [];
const archive = archiver('zip');
archive.on('data', (chunk: Buffer) => chunks.push(chunk));
archive.on('end', () => resolve(Buffer.concat(chunks)));
archive.on('error', reject);
// The runtime requires the script at nodejs/node_modules/<filename>
archive.append(content, { name: `nodejs/node_modules/${filename}` });
archive.finalize();
});
}
Environment variables and secrets
Set environment variables for the canary script via RunConfig.EnvironmentVariables in CreateCanaryCommand. Access them in the script with process.env. For secrets, fetch from Secrets Manager or SSM Parameter Store inside the canary script at runtime — do not put secrets in EnvironmentVariables because they appear in the CloudWatch console and in canary configuration API responses.
// In CreateCanaryCommand — public configuration is fine in EnvironmentVariables
RunConfig: {
TimeoutInSeconds: 60,
EnvironmentVariables: {
TARGET_URL: 'https://api.example.com',
EXPECTED_VERSION: 'v2.1.0',
// For secrets: use SSM or Secrets Manager fetched inside the script,
// not plain values here — these are visible in the CloudWatch console
},
}
// In the canary script — read with process.env
const TARGET_URL = process.env.TARGET_URL ?? 'https://api.example.com';
exports.handler = async () => {
await synthetics.executeStep('check-version', async () => {
// For secrets: use the AWS SDK inside the canary to fetch from SSM/Secrets Manager
// The execution role needs ssm:GetParameter or secretsmanager:GetSecretValue
});
};
Pattern 3: Reading canary run results with GetCanaryRunsCommand
Once canaries are running, MCP server tools can surface current uptime status to users, compute pass rates over time windows, detect outage durations for incident reports, and retrieve the S3 artifact logs when diagnosing a specific failed run. The run results API has two commands: GetCanaryRunsCommand for a single canary's history and DescribeCanariesLastRunCommand for the most recent run across all canaries at once.
Getting recent runs with GetCanaryRunsCommand
Runs are ordered newest-first. Each CanaryRun has three key fields: Status.State (the run outcome), Timeline (start/end timestamps), and ArtifactS3Location (the S3 prefix for log files from this specific run).
import {
SyntheticsClient,
GetCanaryRunsCommand,
type CanaryRun,
} from '@aws-sdk/client-synthetics';
const synthetics = new SyntheticsClient({
region: process.env.AWS_REGION ?? 'us-east-1',
});
export async function getRecentRuns(
canaryName: string,
maxResults = 10,
): Promise<CanaryRun[]> {
const response = await synthetics.send(
new GetCanaryRunsCommand({ Name: canaryName, MaxResults: maxResults })
);
return response.CanaryRuns ?? [];
}
// Example usage
const runs = await getRecentRuns('my-api-canary', 5);
for (const run of runs) {
const durationMs = run.Timeline?.Completed && run.Timeline.Started
? run.Timeline.Completed.getTime() - run.Timeline.Started.getTime()
: null;
console.log({
state: run.Status?.State, // 'PASSED', 'FAILED', or 'RUNNING'
stateReason: run.Status?.StateReason, // error message if FAILED
startedAt: run.Timeline?.Started,
durationMs,
artifactPrefix: run.ArtifactS3Location, // S3 prefix for run logs
});
}
Run state machine
Run states are entirely separate from canary lifecycle states. A canary in RUNNING lifecycle state (active, scheduled) can have its most recent run in FAILED state. Confusing the two is a common source of incorrect status displays.
| Run state | Meaning | Notes |
|---|---|---|
RUNNING | This specific run is currently executing | No completed time yet; ArtifactS3Location may be empty |
PASSED | All steps completed without throwing | Contributes SuccessPercent: 100 to CloudWatch metric |
FAILED | At least one step threw, or timeout exceeded | Check Status.StateReason for the first error message |
Dashboard view: DescribeCanariesLastRunCommand
For a monitoring dashboard that shows the current status of all canaries, DescribeCanariesLastRunCommand is far more efficient than calling GetCanaryRunsCommand once per canary — it returns the most recent run for all canaries in the account/region in a single paginated call.
import {
DescribeCanariesLastRunCommand,
type CanaryLastRun,
} from '@aws-sdk/client-synthetics';
export async function getDashboardStatus(): Promise<
Record<string, { state: string; reason?: string; lastRunAt?: Date; durationMs?: number }>
> {
const results: CanaryLastRun[] = [];
let nextToken: string | undefined;
do {
const response = await synthetics.send(
new DescribeCanariesLastRunCommand({ NextToken: nextToken, MaxResults: 50 })
);
results.push(...(response.CanariesLastRun ?? []));
nextToken = response.NextToken;
} while (nextToken);
return Object.fromEntries(
results.map(entry => {
const run = entry.LastRun;
const startMs = run?.Timeline?.Started?.getTime();
const endMs = run?.Timeline?.Completed?.getTime();
return [
entry.CanaryName ?? 'unknown',
{
state: run?.Status?.State ?? 'UNKNOWN',
reason: run?.Status?.StateReason,
lastRunAt: run?.Timeline?.Completed,
durationMs: startMs && endMs ? endMs - startMs : undefined,
},
];
})
);
}
Outage detection from run history
To compute outage windows for incident reports or SLA calculations, scan run history oldest-first and track contiguous blocks of FAILED runs. This pattern works because GetCanaryRunsCommand returns runs newest-first — reverse the array before scanning.
export async function getOutagePeriods(
canaryName: string,
since: Date,
): Promise<{ start: Date; end: Date | null; durationMinutes: number | null }[]> {
// Collect all runs in the time window
const allRuns: CanaryRun[] = [];
let nextToken: string | undefined;
do {
const response = await synthetics.send(
new GetCanaryRunsCommand({ Name: canaryName, MaxResults: 100, NextToken: nextToken })
);
const runs = response.CanaryRuns ?? [];
const relevantRuns = runs.filter(
r => r.Timeline?.Started && r.Timeline.Started >= since
);
allRuns.push(...relevantRuns);
// Runs are newest-first; stop when we pass our time window
const lastRun = runs[runs.length - 1];
if (lastRun?.Timeline?.Started && lastRun.Timeline.Started < since) break;
nextToken = response.NextToken;
} while (nextToken);
// Process oldest-first to detect contiguous failure windows
const outages = [];
let outageStart: Date | null = null;
for (const run of allRuns.reverse()) {
if (run.Status?.State === 'FAILED' && !outageStart) {
outageStart = run.Timeline?.Started ?? null;
} else if (run.Status?.State === 'PASSED' && outageStart) {
const end = run.Timeline?.Completed ?? null;
outages.push({
start: outageStart,
end,
durationMinutes: end
? Math.round((end.getTime() - outageStart.getTime()) / 60000)
: null,
});
outageStart = null;
}
}
// Open outage (still failing at the end of the time window)
if (outageStart) {
outages.push({ start: outageStart, end: null, durationMinutes: null });
}
return outages;
}
Retrieving run artifacts from S3
The ArtifactS3Location on each run is an S3 URI prefix, not a direct download URL. Parse out the bucket and prefix, list objects under that prefix to find log files, then download the ones you need.
import { S3Client, ListObjectsV2Command, GetObjectCommand } from '@aws-sdk/client-s3';
const s3 = new S3Client({ region: process.env.AWS_REGION ?? 'us-east-1' });
export async function getRunLog(artifactS3Location: string): Promise<string> {
// 's3://bucket/prefix/' → { bucket: 'bucket', prefix: 'prefix/' }
const withoutScheme = artifactS3Location.replace('s3://', '');
const slashIdx = withoutScheme.indexOf('/');
const bucket = withoutScheme.slice(0, slashIdx);
const prefix = withoutScheme.slice(slashIdx + 1);
// The main execution log is at the prefix root
const logKey = `${prefix}canary-run.log`;
const response = await s3.send(new GetObjectCommand({ Bucket: bucket, Key: logKey }));
return response.Body?.transformToString() ?? '';
}
// Your MCP server role needs s3:GetObject and s3:ListBucket on the artifact bucket
// The canary execution role needs s3:PutObject on the same bucket (to write artifacts)
Pattern 4: CloudWatch alarms for canary failures
Rather than polling GetCanaryRunsCommand from your MCP server to detect failures, wire a CloudWatch alarm to the canary's SuccessPercent metric. When the canary starts failing, CloudWatch fires the alarm and publishes to an SNS topic — your server receives a push notification instead of polling. The alarms require no manual metric emission: Synthetics publishes SuccessPercent to the CloudWatchSynthetics namespace automatically after every run.
CloudWatch Synthetics metrics reference
All metrics are published to the CloudWatchSynthetics namespace with the CanaryName dimension. They are emitted after each run completes — not on a fixed time interval, so the metric cadence matches the canary schedule.
| Metric name | Unit | Description |
|---|---|---|
SuccessPercent | Percent | 100 if the run passed, 0 if it failed. Use Average statistic over a window to compute pass rate. |
Duration | Milliseconds | Total execution time for the canary run, from Lambda cold start through all steps completing. |
Failed | Count | 1 for each failed run, 0 for each passed run. Use Sum to count total failures in a window. |
FailedRequests | Count | Number of individual HTTP requests within a run that received non-2xx responses (browser canaries only). |
Creating a failure alarm with PutMetricAlarmCommand
The Period must equal the canary's schedule interval so each evaluation period covers exactly one canary run. A 5-minute canary emits one datapoint every 5 minutes, so Period: 300 gives you one evaluation period per run.
import {
CloudWatchClient,
PutMetricAlarmCommand,
type PutMetricAlarmCommandInput,
} from '@aws-sdk/client-cloudwatch';
const cw = new CloudWatchClient({ region: process.env.AWS_REGION ?? 'us-east-1' });
export async function createCanaryAlarm(params: {
canaryName: string;
snsTopicArn: string;
scheduleIntervalSeconds: number; // e.g. 300 for a 5-minute canary
consecutiveFailuresToAlarm?: number; // default 2
}): Promise<void> {
const n = params.consecutiveFailuresToAlarm ?? 2;
const input: PutMetricAlarmCommandInput = {
AlarmName: `${params.canaryName}-failure-alarm`,
AlarmDescription: `CloudWatch Synthetics canary ${params.canaryName} is failing`,
Namespace: 'CloudWatchSynthetics',
MetricName: 'SuccessPercent',
Dimensions: [{ Name: 'CanaryName', Value: params.canaryName }],
// Period must match the canary schedule interval so each period = 1 run
Period: params.scheduleIntervalSeconds,
Statistic: 'Average', // SuccessPercent is 100 or 0 per run → Average = pass rate
ComparisonOperator: 'LessThanThreshold',
Threshold: 100, // any failure triggers the threshold breach
// Require N consecutive failures before alarming (reduces flapping from blips)
EvaluationPeriods: n,
DatapointsToAlarm: n,
// 'notBreaching': stopped canary = OK (safe default)
// 'breaching': stopped canary = ALARM (use for SLA-critical endpoints)
TreatMissingData: 'notBreaching',
AlarmActions: [params.snsTopicArn], // notify when alarm fires
OKActions: [params.snsTopicArn], // notify when alarm resolves
InsufficientDataActions: [],
};
await cw.send(new PutMetricAlarmCommand(input));
}
Setting up an SNS topic for canary notifications
Create the SNS topic and subscribe notification destinations before wiring it to the alarm. Email subscriptions require manual confirmation from the subscriber; HTTPS subscriptions (Slack webhooks, PagerDuty) auto-confirm if the endpoint returns HTTP 200 to the confirmation POST.
import { SNSClient, CreateTopicCommand, SubscribeCommand } from '@aws-sdk/client-sns';
const sns = new SNSClient({ region: process.env.AWS_REGION ?? 'us-east-1' });
export async function createAlertTopic(params: {
topicName: string;
emailAddresses?: string[];
httpsEndpoints?: string[]; // Slack webhook URLs, PagerDuty endpoints, etc.
}): Promise<string> {
const { TopicArn: topicArn } = await sns.send(
new CreateTopicCommand({ Name: params.topicName })
);
for (const email of params.emailAddresses ?? []) {
// Email subscriptions require the subscriber to click a confirmation link
await sns.send(new SubscribeCommand({
TopicArn: topicArn!,
Protocol: 'email',
Endpoint: email,
}));
}
for (const url of params.httpsEndpoints ?? []) {
// HTTPS subscriptions auto-confirm for Slack/PagerDuty webhook URLs
await sns.send(new SubscribeCommand({
TopicArn: topicArn!,
Protocol: 'https',
Endpoint: url,
}));
}
return topicArn!;
}
// Full setup: create topic + alarm in one call
export async function setupCanaryAlerting(params: {
canaryName: string;
scheduleIntervalSeconds: number;
alertEmails: string[];
slackWebhookUrl?: string;
}) {
const topicArn = await createAlertTopic({
topicName: `${params.canaryName}-alerts`,
emailAddresses: params.alertEmails,
httpsEndpoints: params.slackWebhookUrl ? [params.slackWebhookUrl] : [],
});
await createCanaryAlarm({
canaryName: params.canaryName,
snsTopicArn: topicArn,
scheduleIntervalSeconds: params.scheduleIntervalSeconds,
});
return topicArn;
}
Flapping-resilient alarm variant
The default Threshold: 100 alarms on any single canary failure. For endpoints that have acceptable occasional transient failures, use a lower threshold with a longer evaluation window to alarm only on sustained degradation.
// Alarm if average pass rate drops below 80% over the last 6 runs
export async function createFlappingResilientAlarm(
canaryName: string,
snsTopicArn: string,
scheduleIntervalSeconds: number,
): Promise<void> {
await cw.send(new PutMetricAlarmCommand({
AlarmName: `${canaryName}-degraded-alarm`,
AlarmDescription: `Canary ${canaryName} pass rate below 80% over last 6 runs`,
Namespace: 'CloudWatchSynthetics',
MetricName: 'SuccessPercent',
Dimensions: [{ Name: 'CanaryName', Value: canaryName }],
Period: scheduleIntervalSeconds,
Statistic: 'Average',
ComparisonOperator: 'LessThanThreshold',
Threshold: 80,
EvaluationPeriods: 6,
DatapointsToAlarm: 4, // 4 of 6 periods below 80% = sustained degradation
TreatMissingData: 'notBreaching',
AlarmActions: [snsTopicArn],
OKActions: [snsTopicArn],
}));
}
Composite alarms for multi-canary incident detection
When you have multiple canaries monitoring different aspects of the same service, a composite alarm lets you aggregate them — alarming only when multiple individual alarms fire simultaneously, which reduces noise from isolated endpoint failures that do not indicate a full-service outage.
import { PutCompositeAlarmCommand } from '@aws-sdk/client-cloudwatch';
export async function createCompositeAlarm(params: {
compositeName: string;
childAlarmNames: string[]; // all child alarms must exist before creating composite
snsTopicArn: string;
operator?: 'ANY' | 'ALL'; // ANY = OR, ALL = AND; default: ANY
}): Promise<void> {
const operator = params.operator ?? 'ANY';
const joiner = operator === 'ANY' ? ' OR ' : ' AND ';
const alarmRule = params.childAlarmNames
.map(n => `ALARM("${n}")`)
.join(joiner);
await cw.send(new PutCompositeAlarmCommand({
AlarmName: params.compositeName,
AlarmDescription: `Composite alarm — ${operator} of ${params.childAlarmNames.length} canary alarms`,
AlarmRule: alarmRule,
AlarmActions: [params.snsTopicArn],
OKActions: [params.snsTopicArn],
}));
}
// Example: notify only when BOTH health check and login flow canaries fail
// (reduces false positives from isolated network blips)
await createCompositeAlarm({
compositeName: 'api-platform-down',
childAlarmNames: ['health-check-canary-alarm', 'login-flow-canary-alarm'],
snsTopicArn: 'arn:aws:sns:us-east-1:123456789012:on-call-alerts',
operator: 'ALL',
});
Reading alarm state from an MCP tool
Use DescribeAlarmsCommand to retrieve current alarm state for status dashboards or automated remediation workflows that check whether the alarm is currently in ALARM before taking action.
import {
DescribeAlarmsCommand,
type MetricAlarm,
} from '@aws-sdk/client-cloudwatch';
export async function getAlarmState(alarmName: string): Promise<{
state: string;
reason: string;
updatedAt: Date | undefined;
}> {
const response = await cw.send(
new DescribeAlarmsCommand({ AlarmNames: [alarmName] })
);
const alarm: MetricAlarm | undefined = response.MetricAlarms?.[0];
if (!alarm) throw new Error(`Alarm not found: ${alarmName}`);
return {
// 'OK' | 'ALARM' | 'INSUFFICIENT_DATA'
state: alarm.StateValue ?? 'UNKNOWN',
reason: alarm.StateReason ?? '',
updatedAt: alarm.StateUpdatedTimestamp,
};
}
Alarm states: OK (threshold not breached), ALARM (threshold breached — SNS notification sent), INSUFFICIENT_DATA (no data in the evaluation window — canary may be stopped or newly created). The TreatMissingData setting controls whether INSUFFICIENT_DATA transitions to ALARM or stays INSUFFICIENT_DATA. If your alarm is stuck in INSUFFICIENT_DATA after the canary has been running for several minutes, the first thing to check is whether Namespace and Dimensions[0].Name are spelled exactly as CloudWatchSynthetics and CanaryName respectively — both are case-sensitive.
Related pages
- CloudWatch Synthetics overview — CreateCanaryCommand, canary lifecycle state machine, IAM reference
- Writing CloudWatch Synthetics API canary scripts — Node.js handler structure, synthetics module API, zip packaging
- Reading canary run results with GetCanaryRunsCommand — run state machine, artifact retrieval, outage detection
- CloudWatch alarms for canary failures — SuccessPercent metric, PutMetricAlarmCommand, SNS setup, composite alarms
- CloudWatch overview for MCP servers
- CloudWatch alarm management