Guide · Amazon ElastiCache · Rate Limiting · Lua Scripts · MCP Server Tools

ElastiCache Rate Limiting for MCP Servers — Lua Atomic Scripts, Sliding Window, Token Bucket

MCP server tools that call LLMs, hit external APIs, or run expensive database queries need rate limiting to prevent abuse and control cost. The naive approach — INCR key then EXPIRE key — has a race condition: two concurrent callers both read zero, both increment, and neither sets expiry correctly. The correct approach is Lua scripting: ElastiCache executes a Lua script atomically in a single server round-trip, making the check-increment-expire sequence indivisible. This guide covers every rate limiting pattern you will actually use in production MCP tools — fixed window, sliding window, token bucket — plus cluster mode hash tag co-location, EVALSHA for warm performance, and response headers that callers can consume.

TL;DR

Never use bare INCR + EXPIRE for rate limiting — the two commands are not atomic. Use EVAL with a Lua script that does both in one call. For sliding window accuracy, use a sorted set with ZADD + ZREMRANGEBYSCORE + ZCARD in a single Lua script. For burst-friendly limits, implement token bucket with a Lua script that reads and writes bucket state atomically. In cluster mode, all keys accessed in one EVAL must hash to the same slot — use hash tags like {user:123}:rate. Cache compiled scripts with SCRIPT LOAD + EVALSHA to avoid re-parsing overhead on hot paths.

Why INCR + EXPIRE is broken

The classic mistake looks correct at first glance:

// ---- BROKEN: race condition between INCR and EXPIRE ----
const count = await redis.incr(key);
if (count === 1) {
  await redis.expire(key, windowSeconds); // might never run if process crashes here
}
if (count > limit) {
  throw new Error('rate limit exceeded');
}

Two problems. First, if the process crashes or the connection drops between INCR and EXPIRE, the key has no TTL and persists forever — that user is permanently blocked. Second, two concurrent callers can both read count === 1, both attempt to set expiry, and the window is set twice with slightly different timestamps. Use SET key 0 EX windowSeconds NX to initialize, then INCR — but that is still two round trips. The correct solution is Lua.

Fixed window counter with Lua

A fixed window resets at exact time boundaries (every 60 seconds on the minute). It is the simplest algorithm and the cheapest on memory. The Lua script reads the current count, increments it, sets the TTL only on the first call in a window, and returns both the new count and the remaining TTL so callers can set response headers.

import Redis from 'ioredis';

const redis = new Redis({
  host: process.env.ELASTICACHE_PRIMARY_ENDPOINT,
  port: 6379,
  tls: {},
  password: process.env.ELASTICACHE_AUTH_TOKEN,
});

// Lua script: atomic increment + TTL-on-first-write
const FIXED_WINDOW_SCRIPT = `
local key     = KEYS[1]
local limit   = tonumber(ARGV[1])
local window  = tonumber(ARGV[2])

local count = redis.call('INCR', key)
if count == 1 then
  redis.call('EXPIRE', key, window)
end
local ttl = redis.call('TTL', key)
return { count, ttl }
`;

type RateLimitResult = {
  allowed: boolean;
  count: number;
  remaining: number;
  resetInSeconds: number;
};

async function checkRateLimit(
  identifier: string,          // e.g. userId, IP address
  toolName: string,
  limitPerWindow: number,
  windowSeconds: number,
): Promise {
  const key = `rl:fixed:{${identifier}}:${toolName}`;
  const [count, ttl] = await redis.eval(
    FIXED_WINDOW_SCRIPT, 1, key,
    String(limitPerWindow),
    String(windowSeconds),
  ) as [number, number];

  return {
    allowed: count <= limitPerWindow,
    count,
    remaining: Math.max(0, limitPerWindow - count),
    resetInSeconds: ttl,
  };
}

The key format rl:fixed:{${identifier}}:${toolName} uses a hash tag (the {identifier} part) for cluster mode compatibility — all keys for the same user hash to the same slot regardless of the tool suffix appended outside the braces. See the cluster mode section below.

Fixed window edge case: at the boundary between windows, a user can make 2× their limit in a short burst — limit calls just before the window resets and limit more calls immediately after. If this matters for your use case, use a sliding window instead.

Sliding window counter with Lua

A sliding window measures the last N seconds from now, not from the start of the current window. It prevents the double-burst problem but uses more memory — one sorted set entry per request rather than a single integer counter.

const SLIDING_WINDOW_SCRIPT = `
local key         = KEYS[1]
local now         = tonumber(ARGV[1])
local window_ms   = tonumber(ARGV[2])
local limit       = tonumber(ARGV[3])
local entry_id    = ARGV[4]

-- Remove requests older than the sliding window
redis.call('ZREMRANGEBYSCORE', key, 0, now - window_ms)

-- Count requests in the current window
local count = redis.call('ZCARD', key)

if count < limit then
  -- Add this request with current timestamp as score
  redis.call('ZADD', key, now, entry_id)
  -- TTL = window + 1s buffer to handle clock skew
  redis.call('PEXPIRE', key, window_ms + 1000)
  return { 1, count + 1, limit - count - 1 }
else
  return { 0, count, 0 }
end
`;

async function slidingRateLimit(
  identifier: string,
  toolName: string,
  limit: number,
  windowMs: number,
): Promise<{ allowed: boolean; count: number; remaining: number }> {
  const key = `rl:slide:{${identifier}}:${toolName}`;
  const now = Date.now();
  const entryId = `${now}-${Math.random().toString(36).slice(2, 8)}`;

  const [allowed, count, remaining] = await redis.eval(
    SLIDING_WINDOW_SCRIPT, 1, key,
    String(now), String(windowMs), String(limit), entryId,
  ) as [number, number, number];

  return { allowed: allowed === 1, count, remaining };
}

Memory tradeoff: each request within the window occupies one sorted set entry (approximately 80 bytes). At 100 requests/minute/user with a 60-second window, peak memory per user is about 8 KB. With 10,000 active users, that is 80 MB — budget accordingly. For high-volume tools with relaxed precision requirements, the fixed window is cheaper.

Token bucket for burst-friendly limits

Token bucket allows short bursts up to capacity tokens while enforcing a long-term refill rate of refillRate tokens per second. A user who has been idle accumulates tokens (up to capacity) and can spend them in a burst, then must wait for refill. This maps well to MCP tools called interactively — a user who hasn't made a request in 10 minutes should be able to run a batch of tools immediately.

const TOKEN_BUCKET_SCRIPT = `
local key         = KEYS[1]
local capacity    = tonumber(ARGV[1])
local refill_rate = tonumber(ARGV[2])   -- tokens per second
local now         = tonumber(ARGV[3])   -- current time in seconds (float)
local requested   = tonumber(ARGV[4])   -- tokens to consume (usually 1)

local bucket = redis.call('HMGET', key, 'tokens', 'last_refill')
local tokens      = tonumber(bucket[1])
local last_refill = tonumber(bucket[2])

-- First call: initialize full bucket
if tokens == false or last_refill == false then
  tokens      = capacity
  last_refill = now
end

-- Refill based on elapsed time
local elapsed = now - last_refill
local refilled = elapsed * refill_rate
tokens = math.min(capacity, tokens + refilled)
last_refill = now

if tokens >= requested then
  tokens = tokens - requested
  redis.call('HSET', key, 'tokens', tokens, 'last_refill', last_refill)
  redis.call('EXPIRE', key, math.ceil(capacity / refill_rate) + 60)
  return { 1, math.floor(tokens) }
else
  redis.call('HSET', key, 'tokens', tokens, 'last_refill', last_refill)
  redis.call('EXPIRE', key, math.ceil(capacity / refill_rate) + 60)
  local wait_seconds = (requested - tokens) / refill_rate
  return { 0, math.floor(tokens), math.ceil(wait_seconds) }
end
`;

async function tokenBucketLimit(
  identifier: string,
  toolName: string,
  capacity: number,       // max burst (e.g. 20 tokens)
  refillRate: number,     // tokens per second (e.g. 0.5 = 30/minute)
  tokensRequested = 1,
): Promise<{ allowed: boolean; tokens: number; retryAfterSeconds?: number }> {
  const key = `rl:bucket:{${identifier}}:${toolName}`;
  const now = Date.now() / 1000;

  const result = await redis.eval(
    TOKEN_BUCKET_SCRIPT, 1, key,
    String(capacity), String(refillRate), String(now), String(tokensRequested),
  ) as number[];

  if (result[0] === 1) {
    return { allowed: true, tokens: result[1] };
  }
  return { allowed: false, tokens: result[1], retryAfterSeconds: result[2] };
}

The EXPIRE inside the script keeps the key alive long enough for a fully-depleted bucket to refill. math.ceil(capacity / refill_rate) + 60 gives ceil(20 / 0.5) + 60 = 100 seconds — the bucket completely refills within the TTL window.

EVALSHA — avoiding script re-compilation overhead

Every EVAL call sends the entire Lua script text across the network and forces ElastiCache to parse and compile it. On hot paths (thousands of tool calls per minute), this adds measurable latency. Use SCRIPT LOAD once at startup to upload scripts and get back their SHA1 digests, then use EVALSHA on each call. ElastiCache caches compiled scripts in memory.

// Load scripts at startup and cache their SHAs
const scriptShas: Record = {};

async function loadRateLimitScripts(): Promise {
  scriptShas.fixedWindow  = await redis.script('LOAD', FIXED_WINDOW_SCRIPT);
  scriptShas.slidingWindow = await redis.script('LOAD', SLIDING_WINDOW_SCRIPT);
  scriptShas.tokenBucket  = await redis.script('LOAD', TOKEN_BUCKET_SCRIPT);
  console.log('Rate limit scripts loaded:', scriptShas);
}

// Use EVALSHA instead of EVAL
async function checkRateLimitFast(identifier: string, limit: number, window: number) {
  const key = `rl:fixed:{${identifier}}:default`;
  try {
    const [count, ttl] = await redis.evalsha(
      scriptShas.fixedWindow, 1, key,
      String(limit), String(window),
    ) as [number, number];
    return { allowed: count <= limit, count, ttl };
  } catch (err: any) {
    if (err.message?.includes('NOSCRIPT')) {
      // Script was flushed (server restart) — reload and retry
      await loadRateLimitScripts();
      return checkRateLimitFast(identifier, limit, window);
    }
    throw err;
  }
}

Always catch NOSCRIPT errors. ElastiCache flushes the script cache on node restarts and failovers. After a primary/replica failover (which cluster mode can trigger), the new primary may not have your scripts loaded.

Cluster mode: hash tags and key co-location

Redis Cluster distributes keys across 16,384 hash slots. A Lua EVAL in cluster mode is only valid if all KEYS[n] arguments hash to the same slot — otherwise you get CROSSSLOT Keys in request don't hash to the same slot. See cluster mode guide for slot mechanics.

For rate limiting, all keys for a given user must land in the same slot. Hash tags — the part of a key between { and } — override the slot calculation: only the substring inside braces is hashed. This lets you place multiple related keys in the same slot regardless of their full names.

// All these keys share the same hash slot (hash of "user:abc123")
// even though their full names differ:
const fixedKey     = `rl:fixed:{user:abc123}:search_tool`;
const slidingKey   = `rl:slide:{user:abc123}:search_tool`;
const bucketKey    = `rl:bucket:{user:abc123}:search_tool`;
const metadataKey  = `rl:meta:{user:abc123}:allowlist`;

// BAD: these two keys will likely land in different slots
//      because {user:abc123} and {admin} hash differently
const userKey  = `rl:fixed:{user:abc123}:search_tool`;
const adminKey = `rl:fixed:{admin}:search_tool`;
// A single EVAL touching both = CROSSSLOT error

// GOOD: if you need a cross-user Lua operation,
//       use separate EVAL calls per slot, not one combined call

When using ioredis in cluster mode, pass keyPrefix in the cluster options if all rate limit keys share a prefix — ioredis uses the first key to route the EVAL to the correct node.

Per-tool vs per-user vs global limits

Most production MCP servers need multiple independent limits applied in sequence. A common layering:

async function enforceRateLimits(
  userId: string,
  toolName: string,
): Promise {
  const [userResult, toolResult, globalResult] = await Promise.all([
    // Per-user: 100 tool calls/minute
    checkRateLimitFast(`user:${userId}`, 100, 60),
    // Per-tool per-user: tool-specific sub-limit (e.g. 10 LLM calls/minute)
    checkRateLimitFast(`user:${userId}:tool:${toolName}`, 10, 60),
    // Global: 10,000 calls/minute across all users (protects upstream APIs)
    checkRateLimitFast('global:all', 10_000, 60),
  ]);

  if (!userResult.allowed) {
    throw Object.assign(
      new Error(`Rate limit exceeded: ${userResult.count}/100 per minute`),
      { code: 'RATE_LIMIT_USER', retryAfter: userResult.ttl },
    );
  }
  if (!toolResult.allowed) {
    throw Object.assign(
      new Error(`Tool rate limit exceeded for ${toolName}`),
      { code: 'RATE_LIMIT_TOOL', retryAfter: toolResult.ttl },
    );
  }
  if (!globalResult.allowed) {
    throw Object.assign(
      new Error('Service temporarily at capacity'),
      { code: 'RATE_LIMIT_GLOBAL', retryAfter: globalResult.ttl },
    );
  }
}

Run the per-user and per-tool checks in parallel since they touch different keys. Run the global check in parallel too — if any limit fires, return the most user-friendly error message (not the global one, which is confusing to individual users).

Response headers and retry semantics

Return rate limit state in response headers so callers can implement exponential backoff intelligently. The standard headers are X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset (epoch seconds when the window resets), and Retry-After on a 429 response (seconds to wait).

// In your MCP server tool handler (Express / Fastify / etc.)
app.post('/mcp/tools/:toolName', async (req, res) => {
  const userId = req.auth.userId;
  const toolName = req.params.toolName;

  let rateLimitMeta: RateLimitResult;
  try {
    rateLimitMeta = await checkRateLimit(userId, toolName, 100, 60);
  } catch (err) {
    // Redis unavailable — fail open (allow) or fail closed (deny) based on policy
    // Fail open preferred for user-facing tools; fail closed for cost-sensitive
    console.error('Rate limit check failed:', err);
    // fail open:
    rateLimitMeta = { allowed: true, count: 0, remaining: 100, resetInSeconds: 60 };
  }

  res.set({
    'X-RateLimit-Limit':     '100',
    'X-RateLimit-Remaining': String(rateLimitMeta.remaining),
    'X-RateLimit-Reset':     String(Math.floor(Date.now() / 1000) + rateLimitMeta.resetInSeconds),
  });

  if (!rateLimitMeta.allowed) {
    res.set('Retry-After', String(rateLimitMeta.resetInSeconds));
    return res.status(429).json({
      error: 'Too Many Requests',
      retryAfterSeconds: rateLimitMeta.resetInSeconds,
    });
  }

  // ... execute tool ...
});

Always handle Redis connection failures in the rate limit path. If your rate limiter is unavailable and you fail closed (deny all requests), a brief Redis hiccup takes down your entire service. Prefer fail-open with an alert and circuit breaker for user-facing tools; fail-closed only for genuinely cost-sensitive operations like paid LLM calls.

Common errors and fixes

Error	Cause	Fix
`ERR wrong number of args for 'eval'`	Passing KEYS count as string not number, or mismatched KEYS/ARGV	Pass numkeys as a plain integer, verify KEYS[1] = ARGV array
`NOSCRIPT No matching script`	EVALSHA after server restart flushed script cache	Catch NOSCRIPT, re-run `SCRIPT LOAD`, retry with new SHA
`CROSSSLOT Keys in request don't hash to the same slot`	Multi-key Lua script in cluster mode with keys in different slots	Use hash tags `{identifier}` to force all keys to same slot
Rate counter never resets	INCR + EXPIRE race: process crashed between the two calls	Use Lua EVAL to make increment + expire atomic
Token bucket grants too many tokens	Server time drift between calls; `now` going backward	Use `TIME` command inside Lua script for server-side time: `local t = redis.call('TIME'); local now = t[1] + t[2]/1e6`
High latency on EVAL	Large script text re-parsed on every call	Use SCRIPT LOAD once at startup + EVALSHA on each call