AWS MemoryDB · 2026-10-22 · MemoryDB for Redis arc
AWS MemoryDB for MCP Servers: Durable State, ACL Security, and Cost-Efficient Probe Archives
Most MCP server teams reach for ElastiCache Redis when they need fast in-memory storage, and for the majority of use cases — probe result caches, short-lived session tokens, ephemeral rate-limit windows — that is the right call. But when the state stored in Redis becomes your source of truth — probe verdicts that feed SLA reports, persistent session credentials that cannot be force-expired on failover, rate-limit counters whose resets would cause a billing anomaly — ElastiCache's asynchronous replication model exposes a silent failure mode: writes acknowledged before the primary fails are simply lost on the successor replica. MemoryDB for Redis eliminates this failure mode by committing every write to a Multi-AZ transaction log before acknowledging success, making it suitable as a primary database behind the Redis protocol. This guide synthesises four MemoryDB feature areas into three structural patterns for MCP teams. Pattern 1 covers durability architecture — the exact write-path commitment difference between ElastiCache and MemoryDB, when the cost premium is justified, CDK cluster setup using L1 CfnCluster and CfnACL constructs, cluster-mode connection patterns with ioredis, hash-tag co-location for multi-key operations, and the full failure-mode taxonomy including MOVED errors from node-endpoint connections and CLUSTERDOWN handling during primary election — synthesised from AWS MemoryDB for MCP Servers. Pattern 2 covers ACL security hardening — the Redis 6 ACL permission grammar for scoping commands and keyspaces, minimum-privilege user definitions for probe writers and probe readers, the default user trap (the open-access ACL ships with an unauthenticated default user and must be replaced before production use), zero-downtime password rotation using the dual-password window, TLS enforcement, and CloudWatch alarm design for sustained authentication failures — synthesised from MemoryDB ACL User Management for MCP Servers. Pattern 3 covers operational resilience — automated snapshot configuration with the often-missed finalSnapshotName parameter, manual pre-deployment checkpoint snapshots, cross-region DR via snapshot export and import (the substitute for MemoryDB's lack of live cross-region replication), ElastiCache-to-MemoryDB migration using RDB export and dual-write backfill, and data tiering on r6gd nodes for cost-efficient probe archives where weeks of historical sorted-set data lives on NVMe SSD while the last-24-hours working set stays in DRAM — synthesised from MemoryDB Snapshots for MCP Servers and MemoryDB Data Tiering for MCP Servers.
TL;DR
- Durability architecture — when to choose MemoryDB over ElastiCache: ElastiCache Redis uses asynchronous replication — a write acknowledged to the client may not have reached any replica when the primary fails, and that write is permanently lost. MemoryDB writes to a Multi-AZ transaction log (minimum two AZs) before the ACK, giving RPO=0 across node failures. The RTO is similar (~30–60 seconds while a replica is promoted); the difference is all in the data-loss window. For MCP servers the decision rule: if your Redis state is read-through-cache behaviour (a miss just recomputes or re-fetches from a source-of-truth database), ElastiCache is correct and cheaper. If your Redis state IS the source of truth — probe verdicts used in SLA calculations, session tokens your server issued and tracks, billing counters you cannot reset safely — use MemoryDB. MemoryDB always runs in cluster mode; connect using the
clustercfg.discovery endpoint (not individual node endpoints), and use a cluster-mode-aware client like ioredis Cluster. Co-locate logically related keys using hash tags ({server-slug}:probe:latest,{server-slug}:probe:history) so multi-key operations like MGET and MULTI/EXEC work across the same hash slot. MemoryDB does not support IAM authentication for data-plane operations — use ACL users with passwords stored in Secrets Manager; your ECS task role only needssecretsmanager:GetSecretValue, not any MemoryDB IAM permissions. CDK L1 constructs (CfnCluster,CfnSubnetGroup,CfnACL,CfnUser) are the current path — no L2 constructs as of CDK v2. Critical CDK dependency order: ACL must reach ACTIVE state before the cluster is provisioned; usecluster.addDependency(acl)explicitly or CloudFormation will attempt to create the cluster while the ACL is still CREATING and fail. AliveMCP monitors MemoryDB write paths by issuing canary HSET writes as part of the probe sequence — distinguishing "MCP endpoint responded 200" from "MCP server's durable state store accepted writes" as separate health signals. - ACL security hardening — minimum-privilege users and the default user trap: MemoryDB requires ACL authentication — there is no unauthenticated cluster mode. Every cluster is associated with an ACL (
CfnACL) that contains named users (CfnUser); each user has an authentication string (password in Redis ACL SETUSER format) and an access string (permission grammar scoping commands and keyspaces). The most dangerous default: AWS provides anopen-accessACL that includes adefaultuser with no password and~* &* +@allaccess — anyone who can reach port 6379 can read and write your cluster without credentials. Never useopen-accessin production. Create a custom ACL that does not include thedefaultuser, and reference it in yourCfnCluster. Access string grammar:~patternscopes key access (glob),+commandallows a command,-commanddenies it,&patternallows Pub/Sub channel access. The model is allowlist — commands not listed are denied by default. Minimum-privilege for an MCP probe writer:~{*}:probe:* ~{*}:session:* +hset +hget +hmget +hmset +hgetall +del +expire +ttl +pttl +exists +incr +incrby +decr +zadd +zrangebyscore. Minimum-privilege for a read-only probe reader (dashboard, alert evaluation):~{*}:probe:* ~{*}:session:* +hget +hmget +hgetall +ttl +pttl +exists +zrangebyscore +zcard +zcount. Theonflag in the authentication string is mandatory — a user withoutonwill show ACTIVE in the AWS console but return WRONGPASS on every login. Zero-downtime password rotation uses MemoryDB's dual-password window: add the new password alongside the old (passwords: [oldPassword, newPassword]), deploy new password to all instances, then remove the old. During the window both passwords are valid; no existing connections are dropped. Propagation after an UpdateUser API call is typically under 30 seconds. Always settlsEnabled: trueon the cluster — ACL passwords transmit in plaintext over an unencrypted connection; TLS applies cluster-wide, not per-user. MonitorAuthenticationErrorsCloudWatch metric: a non-zero sustained value after deployment completion indicates a pod was not restarted or a Secrets Manager value does not match the ACL user. AliveMCP monitors your MemoryDB ACL health via this metric — degraded write-path authentication is detected before it affects uptime data accuracy. - Operational resilience — snapshots, cross-region DR, ElastiCache migration, and data tiering: MemoryDB's Multi-AZ transaction log protects against node failure but not against accidental bulk deletes, logic bugs that corrupt keys, or region-level events. Snapshots are the second durability layer. Two critical configuration traps: (1)
snapshotRetentionLimitdefaults to0— automated snapshots are disabled until you set a non-zero value; set at least 7 for production clusters. (2)finalSnapshotNameis optional — without it, a CloudFormation stack deletion or environment teardown permanently destroys the cluster data with no recovery path; always set this for production clusters. Manual snapshots are taken from a replica, do not affect primary write performance, and persist until explicitly deleted (unlike automated snapshots which rotate at the retention window). Cross-region DR on MemoryDB uses snapshot copy + restore rather than live replication (there is no MemoryDB equivalent of ElastiCache Global Datastore). Copy a daily snapshot to an S3 bucket in the DR region, then restore a new cluster from that S3 snapshot ARN; RTO is 20–60 minutes for a 10–100 GB cluster. Migrating from ElastiCache to MemoryDB: export an ElastiCache RDB backup to S3, restore a new MemoryDB cluster from the S3 snapshot ARN, then either accept a maintenance window or implement dual-write + backfill for zero-downtime migration. Data tiering onr6gd.*nodes extends usable cluster capacity by 2–8x by demoting cold keys to NVMe SSD: DRAM reads at ~1µs, SSD reads at ~40–100µs (40–100x slower but orders of magnitude faster than a network round-trip). The tiering decision is LRU-based and transparent to application code. Must-avoid maxmemory policies with tiering:allkeys-lruandallkeys-lfuevict cold SSD keys before warming them, defeating the tiering purpose. Usenoevictionfor archives where all data matters, orvolatile-ttlto assign short-lived keys as sacrificeable and protect archive keys with no TTL. Data tiering break-even: cost advantage begins when total data is 3–4x larger than the working set. For AliveMCP probe archives — working set (last 4 hours of probes) vs archive (last 30 days) ratio of 1:180 — data tiering delivers approximately 75% cost reduction vs DRAM-only r7g nodes. AliveMCP monitors SSD saturation viaStorageBytesUsedat 85% threshold, DRAM pressure viaDatabaseMemoryUsagePercentageat 75%, and latency baseline drift via a 7-day rollingGetTypeCmdsLatencyp99 window — SSD tier saturation typically announces itself 2–3 days before it causes severe degradation. - End-to-end architecture — durable MCP monitoring backend on MemoryDB with data tiering: A complete MemoryDB stack for MCP server monitoring: (1) probe workers write verdicts via ACL
probe-writeruser to{server-slug}:probe:latesthash (HSET, EXPIRE 86400) and append timestamped results to{server-slug}:probe:historysorted set (ZADD, score = Unix timestamp); hash-tag co-location ensures MULTI/EXEC across latest + history works in cluster mode. (2) ACLprobe-readeruser (read-only, same keyspace scope) serves the dashboard and alert-evaluation workers; no path exists for a compromised read service to corrupt probe state. (3) ACLarchiveruser (read-only + SCAN) drains probe history to Postgres on a weekly cadence; SCAN permission explicitly excluded from the probe-writer user to prevent accidental key discovery during writes. (4) Cluster deployed onr6gd.xlargewithdataTiering: 'true': last-24-hours probes stay in DRAM (13.1 GB DRAM for 1,000+ monitored servers' recent data), last-30-days archive on NVMe SSD (237 GB for months of history at median probe frequency). (5) Daily snapshots with 14-day retention andfinalSnapshotNameset. (6) AliveMCP probe sequence: HSET canary write → check acknowledgement → HGET read-back → verify value matches → report MemoryDB write path as healthy or degraded independently of the MCP endpoint health. This architecture survives primary node failure (Multi-AZ transaction log, RPO=0), cluster re-sharding (client-side cluster slot map refresh), and OOM conditions (noeviction on archive keys, volatile-ttl on short-lived counters) without data loss or silent state divergence.
Pattern 1 — Durability Architecture: Choosing MemoryDB and Connecting Correctly
The write-path commitment difference
The difference between ElastiCache Redis and MemoryDB is not visible from application code — both speak the Redis protocol on port 6379. The difference is in the write-path commitment guarantee that determines what happens the instant a primary node fails after acknowledging a write.
ElastiCache Redis replicates writes from the primary to replica nodes asynchronously. The primary acknowledges the write to the client, then propagates it to replicas in the background. If the primary fails between the ACK and the replication, the replica promoted to primary is missing that write. This is documented and accepted behaviour for a cache: the miss rate increases briefly, the cache warms up again from the source-of-truth database, and no functional damage occurs.
MemoryDB's write path is different. Before the primary returns an ACK to the client, it writes to a Multi-AZ transaction log spanning at least two Availability Zones. Only after both AZ acknowledgements does the primary ACK the client. If the primary then fails, the new primary replays the transaction log and has every acknowledged write. RPO is effectively zero on node failure.
| Dimension | ElastiCache Redis | MemoryDB for Redis |
|---|---|---|
| Write durability | Asynchronous replication — some writes lost on failover | Multi-AZ transaction log — RPO=0 |
| Primary use case | Cache, ephemeral state | Primary database with Redis interface |
| Cluster topology | Optional cluster mode | Cluster mode always on (1–500 shards) |
| ACL authentication | AUTH token or ACL (Redis 6.x+) | ACL users required — no plain AUTH |
| Failover RTO | ~30–60 seconds | ~30–60 seconds (similar RTO) |
| Data tiering (SSD) | Not supported | Supported on r6gd nodes |
| Cross-region | Global Datastore (live replication) | Snapshot copy only (no live replication) |
| Price premium | Baseline | ~20–30% higher for equivalent node type |
The decision rule for MCP server teams: if your Redis state is a cache — probe result memos, computed embeddings, API response caches where a miss just re-fetches — ElastiCache is correct and approximately 25% cheaper. If your Redis state is a database — probe verdicts you report in SLA calculations, session tokens your server issues and cannot safely revoke and reissue on failover, rate-limit counters whose reset would cause a billing discrepancy — MemoryDB is the right primitive. The 20–30% price premium buys RPO=0, not lower latency or higher throughput.
Cluster topology: always cluster mode
MemoryDB always runs in cluster mode. A cluster consists of one or more shards; each shard has one primary node and zero to five replicas. Data is distributed across 16,384 hash slots, split evenly across your shards. With two shards, shard 1 owns hash slots 0–8191 and shard 2 owns 8192–16383.
The cluster discovery endpoint — identified by the clustercfg. prefix in its DNS name — is what your application connects to. The client sends a CLUSTER SLOTS or CLUSTER SHARDS command on startup and builds an internal slot-to-node map. When you run a command, the client hashes the key to a slot, looks up which node owns that slot, and routes the command directly. When a failover or re-shard moves slots, the client receives a MOVED redirect and updates its slot map.
The most common connection mistake: connecting to an individual node endpoint instead of the cluster endpoint. Individual node endpoints exist and are valid DNS names, but they bypass cluster routing. A command sent to the wrong node returns MOVED forever rather than redirecting correctly. Connecting to a node endpoint with a cluster-mode-unaware client produces the MOVED error and breaks the connection. Always use the clustercfg. endpoint.
CDK cluster setup
MemoryDB L1 constructs are in aws-cdk-lib/aws-memorydb. The dependency order matters: the ACL must reach ACTIVE state before the cluster is provisioned.
import * as cdk from 'aws-cdk-lib';
import { aws_memorydb as memorydb, aws_ec2 as ec2 } from 'aws-cdk-lib';
const subnetGroup = new memorydb.CfnSubnetGroup(this, 'MemoryDbSubnetGroup', {
subnetGroupName: 'mcp-probes-memorydb',
subnetIds: vpc.selectSubnets({ subnetType: ec2.SubnetType.PRIVATE_ISOLATED }).subnetIds,
});
const memorydbSg = new ec2.SecurityGroup(this, 'MemoryDbSg', { vpc });
memorydbSg.addIngressRule(mcpServerSg, ec2.Port.tcp(6379), 'MemoryDB port from MCP server');
const acl = new memorydb.CfnACL(this, 'MemoryDbAcl', {
aclName: 'mcp-probes-acl',
userNames: ['mcp-probe-writer', 'mcp-probe-reader'],
});
const cluster = new memorydb.CfnCluster(this, 'MemoryDbCluster', {
clusterName: 'mcp-probes',
nodeType: 'db.r7g.large',
numShards: 2,
numReplicasPerShard: 1,
subnetGroupName: subnetGroup.subnetGroupName,
securityGroupIds: [memorydbSg.securityGroupId],
aclName: acl.aclName,
tlsEnabled: true,
autoMinorVersionUpgrade: true,
snapshotRetentionLimit: 7,
snapshotWindow: '05:00-06:00',
maintenanceWindow: 'sun:07:00-sun:08:00',
parameterGroupName: 'default.memorydb-redis7',
engineVersion: '7.1',
finalSnapshotName: 'mcp-probes-final', // critical: without this, stack deletion = data loss
});
cluster.addDependency(subnetGroup);
cluster.addDependency(acl); // ACL must be ACTIVE before cluster creation starts
Connecting from MCP server code
Use ioredis in cluster mode. The cluster endpoint's clustercfg. prefix signals to ioredis that this is a cluster discovery endpoint, not a single-node endpoint.
import Redis from 'ioredis';
const redisClient = new Redis.Cluster(
[{ host: process.env.MEMORYDB_ENDPOINT, port: 6379 }],
{
dnsLookup: (address, callback) => callback(null, address),
redisOptions: {
tls: {},
username: 'mcp-probe-writer',
password: process.env.MEMORYDB_PASSWORD,
},
enableReadyCheck: true,
scaleReads: 'slave', // distribute reads across replicas
clusterRetryStrategy: (times) => Math.min(100 * 2 ** times, 3000),
}
);
// Hash tag co-location: keys with the same tag in {…} land in the same hash slot
// This enables MGET, MULTI/EXEC, and pipeline across related keys for the same server
await redisClient.hset('{mcp-server-slug}:probe', 'status', 'up', 'ts', Date.now());
await redisClient.expire('{mcp-server-slug}:probe', 86400);
await redisClient.zadd(
'{mcp-server-slug}:probe:history',
Date.now() / 1000,
JSON.stringify({ status: 'up', latencyMs: 38 })
);
Hash tags ensure that {mcp-server-slug}:probe and {mcp-server-slug}:probe:history land in the same hash slot, making MULTI/EXEC transactions across them legal in cluster mode. Without hash tags, a transaction spanning two keys owned by different shards returns a CROSSSLOT error.
Connection failure taxonomy
| Error | Cause | Fix |
|---|---|---|
| WRONGPASS | ACL user password incorrect or user not in active ACL | Verify Secrets Manager value matches CfnUser; confirm user is in the cluster's ACL |
| NOPERM | ACL user's access string doesn't permit the command or keyspace | Expand access string in CfnUser; wait for ACL to reach ACTIVE after update |
| MOVED | Client not in cluster mode — connected to node endpoint or using non-cluster client | Use clustercfg. endpoint with ioredis Cluster; never connect to individual node endpoints |
| CROSSSLOT | Multi-key command spans keys in different hash slots | Add hash tags {…} so related keys co-locate; restructure multi-key commands |
| CLUSTERDOWN | Cluster in primary election — transient during failover (30–60s) | Retry with exponential backoff; do not alert until past 90-second recovery threshold |
| TLS handshake failure | Client TLS disabled or security group blocks port 6379 | Enable tls: {} in ioredis; verify SG inbound rule from MCP server SG |
How AliveMCP validates the MemoryDB write path
AliveMCP's probe sequence for MCP servers backed by MemoryDB includes a canary write phase: the probe issues an HSET to a reserved canary key in your cluster, verifies the write was acknowledged, then reads it back with HGET and checks the value matches. This distinguishes "MCP endpoint returned HTTP 200" from "MCP server's durable state store accepted writes" — two separate health signals that can diverge when MemoryDB is under CLUSTERDOWN pressure or when ACL authentication failures start accumulating.
The canary key uses the server slug as hash tag so it lands in the same shard as the real probe state for that server, ensuring the canary exercises the same primary node path. A probe-writer authentication failure surfaces immediately in the canary phase before any real probe state is affected.
Pattern 2 — ACL Security Hardening: Minimum Privilege and the Default User Trap
How MemoryDB ACLs are structured
MemoryDB's ACL model differs from self-managed Redis in one critical structural way: users are defined as AWS resources (CfnUser), grouped into an ACL resource (CfnACL), and the ACL is associated with a cluster at the CDK/CloudFormation level. You do not configure ACLs on the cluster directly via Redis commands — you update the AWS resource and wait for propagation.
This indirection matters for operations teams: adding a new user to an existing cluster requires updating the CfnACL's userNames list (CloudFormation update), which transitions the ACL through CREATING → MODIFYING → ACTIVE states. The MODIFYING state is not an error — it is expected during routine ACL updates, and existing connections work normally during propagation. If an ACL stays in MODIFYING for more than 15 minutes, check whether the associated cluster is itself in UPDATING state — a cluster update can hold an ACL in MODIFYING until the cluster update completes.
The default user trap
AWS provides a built-in open-access ACL with a default user configured as on nopass ~* &* +@all: no password required, full access to all keys and commands. This ACL is included in new clusters by default. It exists to simplify development and must be replaced before production use.
The failure mode is subtle: the cluster passes all connectivity tests (authentication works because no auth is required), but any IP that can reach port 6379 can read and write your probe state or session tokens. Your VPC security group is the only protection, and any misconfigured SG inbound rule — or an overly permissive peered VPC — exposes the cluster without any credential challenge.
Replace the open-access ACL at cluster creation with a custom ACL that does not include the default user. The default user exists at the AWS account level but is inert when not included in an active cluster ACL.
Access string grammar for minimum-privilege users
Access strings follow Redis 6 ACL SETUSER permission grammar. The key constructs:
~pattern— glob pattern scoping key access. Multiple patterns are OR'd.~*allows all keys. Hash-tag patterns work:~{*}:probe:*matches any key whose hash tag prefix is{something}:probe:.+command— allows a command.+@readallows all read commands;+@hashallows all hash commands. Use category shortcuts sparingly — they allow future Redis commands added to the category automatically.-command— denies a specific command. Useful to deny dangerous commands after a broad allow:+@all -@dangerous -@admin -debug.&pattern— allows Pub/Sub channel access. Required only if the user calls SUBSCRIBE, PUBLISH, or PSUBSCRIBE.- Allowlist model: commands not listed are denied. The list starts empty for a new user.
# Probe writer: writes verdicts and history; no read of other servers' keys; no admin
"on resetpassword >ProbeWriterP@ss ~{*}:probe:* ~{*}:session:* +hset +hget +hmget +hmset +hgetall +del +expire +ttl +pttl +exists +incr +incrby +decr +zadd +zrangebyscore"
# Probe reader (dashboard, alert evaluation): read-only, same keyspace
"on resetpassword >ProbeReaderP@ss ~{*}:probe:* ~{*}:session:* +hget +hmget +hgetall +ttl +pttl +exists +zrangebyscore +zcard +zcount"
# Archiver: read + SCAN for key discovery; explicitly no writes
"on resetpassword >ArchiverP@ss ~{*}:probe:* +hget +hmget +hgetall +zrangebyscore +zcard +scan"
# Admin (rotation function only — keep offline except during rotation):
"on resetpassword >AdminP@ss ~* &* +@all"
The on flag is mandatory — a user without on in the authentication string will appear ACTIVE in the AWS console but return WRONGPASS on every connection attempt. This is the single most common MemoryDB ACL misconfiguration in CDK deployments: the user resource creates successfully, but the cluster rejects it as disabled.
CDK user and ACL setup with Secrets Manager
import { aws_memorydb as memorydb, aws_secretsmanager as sm } from 'aws-cdk-lib';
const probeWriterSecret = new sm.Secret(this, 'ProbeWriterSecret', {
secretName: '/mcp-probes/memorydb/probe-writer',
generateSecretString: {
excludePunctuation: true, // Redis passwords cannot contain spaces
passwordLength: 32,
},
});
const probeWriterUser = new memorydb.CfnUser(this, 'ProbeWriterUser', {
userName: 'mcp-probe-writer',
accessString: '~{*}:probe:* ~{*}:session:* +hset +hget +hmget +hmset +hgetall +del +expire +ttl +pttl +exists +incr +incrby +decr +zadd +zrangebyscore',
authenticationMode: {
type: 'password',
passwords: [probeWriterSecret.secretValue.unsafeUnwrap()],
},
});
const probeReaderSecret = new sm.Secret(this, 'ProbeReaderSecret', {
secretName: '/mcp-probes/memorydb/probe-reader',
generateSecretString: { excludePunctuation: true, passwordLength: 32 },
});
const probeReaderUser = new memorydb.CfnUser(this, 'ProbeReaderUser', {
userName: 'mcp-probe-reader',
accessString: '~{*}:probe:* ~{*}:session:* +hget +hmget +hgetall +ttl +pttl +exists +zrangebyscore +zcard +zcount',
authenticationMode: {
type: 'password',
passwords: [probeReaderSecret.secretValue.unsafeUnwrap()],
},
});
// ACL: probe users only — default user NOT included
const acl = new memorydb.CfnACL(this, 'MemoryDbAcl', {
aclName: 'mcp-probes-acl',
userNames: [probeWriterUser.userName, probeReaderUser.userName],
});
Zero-downtime password rotation
MemoryDB supports up to two active passwords per user. The rotation pattern uses this dual-password window to avoid any connection interruption:
- Generate new password, write to Secrets Manager at a new version
- Add new password to the user alongside the old:
passwords: [oldPassword, newPassword]. Both are valid immediately after propagation (~30 seconds). - Deploy new password to all MCP server instances — update the Secrets Manager reference, trigger a rolling restart or connection pool refresh
- Remove the old password:
passwords: [newPassword]
# Step 2: add new password (dual-password window)
aws memorydb update-user \
--user-name mcp-probe-writer \
--authentication-mode '{"Type":"password","Passwords":["oldPassword","newPassword"]}'
# Step 4: remove old password after all instances use new
aws memorydb update-user \
--user-name mcp-probe-writer \
--authentication-mode '{"Type":"password","Passwords":["newPassword"]}'
The Lambda rotation function in Secrets Manager can automate this pattern via the MemoryDB UpdateUser API. Existing connections are not dropped during the update — MemoryDB propagates asynchronously and the current connection's auth state is not re-evaluated mid-session.
CloudWatch alarm for authentication failures
import { aws_cloudwatch as cloudwatch } from 'aws-cdk-lib';
new cloudwatch.Alarm(this, 'MemoryDbAuthErrorAlarm', {
metric: new cloudwatch.Metric({
namespace: 'AWS/MemoryDB',
metricName: 'AuthenticationErrors',
dimensionsMap: { ClusterName: 'mcp-probes' },
statistic: 'Sum',
period: cdk.Duration.minutes(5),
}),
threshold: 10,
evaluationPeriods: 3, // 15 consecutive minutes of errors
datapointsToAlarm: 3,
comparisonOperator: cloudwatch.ComparisonOperator.GREATER_THAN_THRESHOLD,
alarmDescription: 'MemoryDB auth errors sustained — check Secrets Manager / ACL user state',
treatMissingData: cloudwatch.TreatMissingData.NOT_BREACHING,
});
A brief non-zero AuthenticationErrors count during deployments is normal — old pods reconnecting with the previous password before rotation completes. A sustained non-zero value after deployment stabilises indicates either a pod was not restarted, a Secrets Manager value is stale, or an ACL update propagation is stuck.
Pattern 3 — Operational Resilience: Snapshots, DR, Migration, and Data Tiering
The two gaps that snapshots fill
MemoryDB's Multi-AZ transaction log covers node failure and AZ outages. It does not cover:
- Logical corruption: a bug deploys that calls DEL on the wrong key prefix and wipes probe state for 200 servers. The transaction log faithfully replays the DEL commands — the data is gone across all replicas.
- Region-level events: in a region-wide outage, the cluster is unavailable and the transaction log cannot be accessed until the region recovers.
Snapshots address both. Automated daily snapshots provide operational recovery (roll back to yesterday's state after a logic bug). Cross-region snapshot copies provide DR (restore in a second region during a regional event).
Automated snapshot configuration — two frequently missed parameters
const cluster = new memorydb.CfnCluster(this, 'MemoryDbCluster', {
clusterName: 'mcp-probes',
// ...
snapshotRetentionLimit: 14, // DEFAULT IS 0 — automated snapshots DISABLED without this
snapshotWindow: '05:00-06:00', // UTC window; avoid overlap with maintenanceWindow
maintenanceWindow: 'sun:07:00-sun:08:00',
finalSnapshotName: 'mcp-probes-final', // without this, stack delete = permanent data loss
});
snapshotRetentionLimit = 0 is the default. A cluster deployed without setting this parameter has no automated backups. Free snapshot storage equals one snapshot's worth of data (the cluster's total data size); additional snapshots are charged at S3 rates (~$0.023/GB-month). For a 100 GB cluster with 14 daily snapshots, the 13 additional snapshots cost ~$30/month — modest relative to cluster costs.
The snapshot window and maintenance window cannot overlap: MemoryDB will defer one if they conflict. Choose a 1-hour snapshot window during low-traffic off-hours and a separate 1-hour maintenance window. If they overlap, you get silent deferral — snapshots appear to run but actually skip days, which is invisible until you check the snapshot count.
Manual snapshots for pre-deployment checkpoints
# Take before any deployment that writes to probe state
aws memorydb create-snapshot \
--cluster-name mcp-probes \
--snapshot-name mcp-probes-pre-deploy-$(date +%Y-%m-%d-%H%M)
# Wait for Available status (5–15 minutes for 10 GB cluster)
aws memorydb describe-snapshots \
--snapshot-name mcp-probes-pre-deploy-2026-10-22-1430 \
--query 'Snapshots[0].{Status:Status,CreationTime:CreationTime}'
Manual snapshots are taken from a replica node and do not affect primary write performance. They persist until explicitly deleted — unlike automated snapshots, they are not subject to the retention window. Clean up pre-deployment snapshots after the deployment is confirmed stable to avoid accumulating storage costs.
Restoring from a snapshot
MemoryDB snapshot restore creates a new cluster — there is no in-place restore that overwrites an existing cluster's data. The restore workflow:
- Provision a new cluster pointing to the snapshot
- Update the MCP server's connection endpoint (
MEMORYDB_ENDPOINTenv var, restart ECS tasks) - Verify data integrity against the corrupted-state cluster
- Delete the corrupted cluster (with a final snapshot as a recovery point)
aws memorydb create-cluster \
--cluster-name mcp-probes-restored \
--node-type db.r7g.large \
--acl-name mcp-probes-acl \ # must pre-exist and be ACTIVE
--subnet-group-name mcp-probes-memorydb \
--security-group-ids sg-0123456789abcdef \
--tls-enabled \
--snapshot-name mcp-probes-pre-deploy-2026-10-22-1430 \
--num-shards 2 \
--num-replicas-per-shard 1
Restore constraints: you can restore to the same or more shards (not fewer); you cannot restore a Redis 7.1 snapshot to a Redis 6.2 cluster (upgrades yes, downgrades no); the ACL must pre-exist in ACTIVE state (it is not cloned from the source cluster). MemoryDB does not support point-in-time restore (unlike RDS) — recovery is to the snapshot time, not an arbitrary minute between snapshots.
Cross-region DR via snapshot copy
MemoryDB does not support live cross-region replication (no equivalent to ElastiCache Global Datastore). Cross-region DR uses snapshot export and import:
# Step 1: copy snapshot to S3 bucket in destination region
aws memorydb copy-snapshot \
--source-snapshot-name mcp-probes-daily-2026-10-22 \
--target-snapshot-name mcp-probes-us-west-2-2026-10-22 \
--target-bucket arn:aws:s3:::mcp-probes-dr-snapshots-us-west-2 \
--kms-key-id arn:aws:kms:us-west-2:123456789:key/dr-key
# Step 2: restore in destination region from S3 snapshot
aws memorydb create-cluster \
--region us-west-2 \
--cluster-name mcp-probes-dr \
--node-type db.r7g.large \
--acl-name mcp-probes-acl \
--subnet-group-name mcp-probes-memorydb \
--security-group-ids sg-abcdef1234567890 \
--tls-enabled \
--snapshot-arns arn:aws:s3:::mcp-probes-dr-snapshots-us-west-2/mcp-probes-us-west-2-2026-10-22.rdb \
--num-shards 2 --num-replicas-per-shard 1
RTO for cross-region restore is 20–60 minutes for a 10–100 GB cluster. For MCP servers requiring sub-5-minute cross-region RTO, MemoryDB is not the right primitive — consider DynamoDB Global Tables or a multi-region write architecture with a writable copy in each region. MemoryDB cross-region DR is suited for monitoring-tier data (probe state, uptime history) where a 30-minute recovery window is operationally acceptable.
Migrating from ElastiCache to MemoryDB
The common upgrade path: an MCP server initially stores probe state in ElastiCache Redis (acceptable as cache), the monitoring use case grows to require SLA reporting, and the team needs RPO=0 durability without changing application code. The migration path is RDB export from ElastiCache + MemoryDB import.
| Migration approach | Downtime | Risk | Use when |
|---|---|---|---|
| RDB import with maintenance window | Yes (minutes to hours) | Low | Non-critical state, flexible SLA |
| Dual-write + backfill | No | Medium (backfill complexity) | Critical state, zero-downtime requirement |
| Application-level replay from event log | No | Low (requires event sourcing) | Event-sourced architectures |
# Export ElastiCache RDB to S3 (requires backup enabled on ElastiCache cluster)
# Then import into MemoryDB
aws memorydb create-cluster \
--cluster-name mcp-probes-migrated \
--node-type db.r7g.large \
--acl-name mcp-probes-acl \
--subnet-group-name mcp-probes-memorydb \
--security-group-ids sg-0123456789abcdef \
--tls-enabled \
--snapshot-arns arn:aws:s3:::my-backup-bucket/elasticache-export/my-cluster.rdb \
--num-shards 2 --num-replicas-per-shard 1
For zero-downtime migration: import the ElastiCache RDB into MemoryDB (catches up to the export time), enable dual-write in the application (all writes go to both), backfill the delta from the write-after-export window, switch reads to MemoryDB, remove the ElastiCache write path. The dual-write phase adds latency because both writes must succeed before the operation is acknowledged — size this window based on your write volume and target migration completion time.
Data tiering on r6gd nodes — the probe archive cost model
MemoryDB data tiering is available only on r6gd.* node types, which have both DRAM and NVMe SSD. The tiering engine is LRU-based and transparent to application code: recently accessed keys live in DRAM (~1µs read latency), cold keys are demoted to SSD (~40–100µs read latency), and SSD keys are promoted to DRAM on next access. Write path always goes to DRAM first; demotion to SSD happens asynchronously.
| Node type | DRAM | NVMe SSD | Tiering |
|---|---|---|---|
| db.r7g.large | 13.1 GB | None | No |
| db.r6gd.large | 13.1 GB | 118 GB | Yes |
| db.r6gd.xlarge | 26.3 GB | 237 GB | Yes |
| db.r6gd.2xlarge | 52.6 GB | 474 GB | Yes |
| db.r6gd.4xlarge | 105 GB | 950 GB | Yes |
The SSD:DRAM ratio is approximately 9:1 — for every GB of DRAM, ~9 GB of NVMe SSD. The cost break-even is when total data volume is 3–4x larger than the working set. Below that ratio, DRAM-only r7g nodes are cheaper per GB and deliver lower, more predictable latency.
maxmemory-policy interaction with data tiering
The maxmemory-policy controls what happens when DRAM fills up. With tiering, the wrong policy defeats the purpose of paying for SSD storage:
- Avoid with tiering:
allkeys-lruandallkeys-lfu— these evict the least-recently-used keys from the entire dataset including SSD. When DRAM fills, the LRU candidates are the cold SSD keys — they get evicted before being promoted. You pay for SSD storage but the engine deletes its contents rather than serving them. - Use with tiering:
noevictionfor archives where every data point matters — MemoryDB returns OOM on new writes rather than evicting old data; size DRAM to hold your working set.volatile-ttlfor mixed workloads — assign TTLs only to expendable keys (rate counters, short-lived session windows); archive keys have no TTL and are never evicted; under memory pressure the short-lived keys are evicted first by TTL proximity.
// CDK custom parameter group for data-tiering archive workload
const paramGroup = new memorydb.CfnParameterGroup(this, 'DataTieringParamGroup', {
family: 'memorydb_redis7',
parameterGroupName: 'mcp-probe-archives-params',
parameters: {
'maxmemory-policy': 'volatile-ttl', // protect archive keys (no TTL) from eviction
'hz': '100', // faster TTL expiry for short-lived probe windows
'lazyfree-lazy-eviction': 'yes', // async eviction — avoid latency spikes
'lazyfree-lazy-expire': 'yes',
},
});
The dataTiering: 'true' flag must be set explicitly on the CfnCluster even on r6gd nodes — without it, only DRAM is used and the NVMe SSD capacity is wasted.
AliveMCP probe archive access pattern for data tiering
// Write probe result: always ZADD (append-only — no score updates to cold entries)
await redis.zadd(
`{${serverSlug}}:probes:history`,
Date.now() / 1000,
JSON.stringify({ status: 'up', latencyMs: 42, ts: Date.now() })
);
// Dashboard query (last 24 hours): hits DRAM tier — fast
const recent = await redis.zrangebyscore(
`{${serverSlug}}:probes:history`,
Date.now() / 1000 - 86400,
'+inf'
);
// SLA report (last 30 days): promotes keys from SSD on first access
// Subsequent dashboard loads of same period hit DRAM (promoted earlier)
const monthly = await redis.zrangebyscore(
`{${serverSlug}}:probes:history`,
Date.now() / 1000 - 2592000,
'+inf'
);
The append-only pattern is critical: ZADD with a new timestamp score always adds a new member. If you update existing members' scores (e.g., incrementing a historical counter), each update promotes the target key from SSD to DRAM — on a high-throughput server this becomes severe read amplification on the SSD tier. For probe archives, write new entries only; never update historical scores.
Monitoring tier health
| CloudWatch metric | Alert threshold | What it signals |
|---|---|---|
| StorageBytesUsed | 85% of SSD capacity | SSD tier approaching saturation — scale up or add shards before OOM |
| DatabaseMemoryUsagePercentage | 75% of DRAM | DRAM pressure — tiering engine hyperactive; working set may be too large for DRAM tier |
| GetTypeCmdsLatency (p99) | 2x 7-day rolling baseline (or 5ms absolute) | SSD tier thrashing — DRAM miss rate elevated, promotions causing latency |
| EngineCPUUtilization | 80% | Tiering promotion CPU overhead becoming a bottleneck |
| CurrItems | Rate-of-growth projection | Project SSD exhaustion date based on growth rate |
SSD saturation causes severe performance degradation — MemoryDB either starts evicting from SSD (if maxmemory-policy allows) or returns OOM errors (noeviction). The saturation trajectory is usually visible 2–3 days before the problem becomes acute, via a steady upward trend in StorageBytesUsed. Set the 85% alarm and act on it before reaching 95%.
End-to-End: Durable MCP Monitoring Stack on MemoryDB
Combining the three patterns produces a complete, operationally sound MemoryDB deployment for MCP server probe state and uptime archives:
Architecture overview
- Cluster selection: for probe state + session tokens (source of truth), use MemoryDB on r6gd.xlarge with data tiering enabled. For probe result caches (ephemeral, re-fetchable), keep ElastiCache Redis — cheaper and lower latency for the cache use case.
- Shard topology: 2 shards, 1 replica per shard. Hash-tag all keys by server slug:
{server-slug}:probe:latest,{server-slug}:probe:history,{server-slug}:session:*. All operations for one server are within one shard — MULTI/EXEC and MGET work without cross-slot errors. - ACL users:
probe-writer(write + read probe keyspace),probe-reader(read-only probe keyspace),archiver(read + SCAN, no write path). Nodefaultuser. Passwords in Secrets Manager with dual-password rotation. TLS enforced at the cluster level. - Snapshots: 14-day automated retention, snapshot window 05:00–06:00 UTC, maintenance window 07:00–08:00 UTC Sunday (non-overlapping).
finalSnapshotNameset for protection against accidental stack deletion. Manual snapshot before any probe-state-modifying deployment. - Data tiering:
maxmemory-policy: volatile-ttl. Archive sorted-set keys (probe history) have no TTL — never evicted. Short-lived hash keys (latest probe state, session tokens) have 86400s TTL — evicted under DRAM pressure before archive data is touched. - AliveMCP monitoring: canary HSET/HGET write probe on each session; AuthenticationErrors CloudWatch alarm; StorageBytesUsed at 85%; GetTypeCmdsLatency at 2x baseline; DRAM usage at 75%.
Data flow
Probe worker (ECS task, probe-writer ACL user) runs every minute per monitored MCP server:
- HTTP probe to MCP endpoint → record latency and status
HSET {slug}:probe:latest status up latencyMs 38 ts 1761148200+EXPIRE 86400ZADD {slug}:probe:history 1761148200 '{"status":"up","latencyMs":38}'(no TTL — archive key)- MemoryDB Multi-AZ transaction log commits both writes before ACK — state survives any single-node failure
Dashboard service (probe-reader ACL user):
HGETALL {slug}:probe:latest— last-known status from DRAM (sub-millisecond)ZRANGEBYSCORE {slug}:probe:history [now-86400] +inf— last 24 hours from DRAMZRANGEBYSCORE {slug}:probe:history [now-2592000] +inf— 30-day SLA window: first call promotes from SSD (~40–100µs per key), subsequent calls from DRAM
Archiver service (archiver ACL user, weekly):
- SCAN for
*:probe:historykeys - ZRANGEBYSCORE for data older than 30 days
- Write to Postgres time-series table
- ZREMRANGEBYSCORE to delete archived range from MemoryDB SSD
This architecture is self-sustaining: the archiver drains old data from SSD, preventing StorageBytesUsed from growing unbounded; the probe writer continuously adds new entries to the DRAM hot tier; the dashboard reads recent data from DRAM and historical data from SSD on first access. AliveMCP monitors the entire stack — write path health, ACL authentication, DRAM/SSD utilisation, and latency baseline — as a single infrastructure health signal that surfaces independently of the MCP endpoint HTTP status.
Further reading
- AWS MemoryDB for MCP Servers — complete overview: MemoryDB vs ElastiCache, CDK setup, connection patterns
- MemoryDB ACL User Management — access string grammar, default user trap, rotation, TLS enforcement
- MemoryDB Snapshots for MCP Servers — automated backups, restore, cross-region DR, ElastiCache migration
- MemoryDB Data Tiering — r6gd nodes, DRAM/SSD tiers, maxmemory-policy, cost model
- AWS S3 Advanced Patterns for MCP Servers — Event Notifications, Object Lambda, Access Points, Replication