MCP Server Step Functions Map State — parallel tool fan-out, maxConcurrency, and Distributed Map

Inline Map vs Distributed Map: choosing based on item count and result size

The Map state mode is set with the "Type": "Map" field and an optional "Mode": "DISTRIBUTED" field. Inline Map is the default when Mode is absent.

Inline Map runs all iterations within the parent execution's event history. Each iteration generates ~3–10 execution events (depending on the sub-workflow complexity). A 100-item Inline Map with a 10-state iterator generates ~1,500 events — well within the 25,000-event ceiling. But a 1,000-item Inline Map with the same iterator generates ~15,000 events, leaving only 10,000 events for the rest of the workflow. For MCP tools that process large lists, Inline Map is safe up to ~40–50 items with complex iterators or ~200 items with simple (2–3 state) iterators.

Distributed Map runs each iteration as an isolated child execution. Parent execution history records only Map-level events (MapRunStarted, MapRunSucceeded, MapRunFailed), not iteration-level events. Child executions have their own 25,000-event budget. Distributed Map supports reading directly from S3 (JSON Lines, CSV, Amazon States Language JSON arrays) and DynamoDB. ResultWriter writes each child execution's output to S3 as a JSON Lines file, preventing the parent execution from needing to store all results in memory.

// Inline Map ASL snippet (embedded in state machine definition)
const inlineMapState = {
  "Type": "Map",
  "MaxConcurrency": 10,           // max 10 concurrent iterations
  "ToleratedFailurePercentage": 5, // fail Map state only if >5% of items fail
  "ItemsPath": "$.toolCallItems", // array in execution input
  "ItemSelector": {               // transform each item before passing to iterator
    "item.$": "$$.Map.Item.Value",
    "index.$": "$$.Map.Item.Index",
    "executionId.$": "$$.Execution.Id"
  },
  "Iterator": {
    "StartAt": "ProcessItem",
    "States": {
      "ProcessItem": {
        "Type": "Task",
        "Resource": "arn:aws:lambda:us-east-1:123:function:process-tool-item",
        "Retry": [{ "ErrorEquals": ["States.TaskFailed"], "MaxAttempts": 2, "BackoffRate": 2 }],
        "Catch": [{ "ErrorEquals": ["States.ALL"], "Next": "HandleItemFailure", "ResultPath": "$.error" }],
        "End": true
      },
      "HandleItemFailure": { "Type": "Pass", "Result": {"status": "failed"}, "End": true }
    }
  },
  "ResultPath": "$.mapResults",
  "Next": "AggregateResults"
};

// Distributed Map ASL snippet for S3 input and S3 result output
const distributedMapState = {
  "Type": "Map",
  "Mode": "DISTRIBUTED",           // required for Distributed Map
  "MaxConcurrency": 100,
  "ToleratedFailureCount": 10,     // absolute count alternative to percentage
  "ItemReader": {                   // read items from S3 instead of execution input
    "Resource": "arn:aws:states:::s3:getObject",
    "ReaderConfig": {
      "InputType": "JSON",          // or CSV, JSONL
      "MaxItems": 100000            // safety limit
    },
    "Parameters": {
      "Bucket": "my-tool-input-bucket",
      "Key.$": "$.inputKey"
    }
  },
  "ItemBatcher": {                  // group items into batches before sending to iterator
    "MaxItemsPerBatch": 25,         // each iterator invocation receives up to 25 items
    "MaxInputBytesPerBatch": 204800 // 200KB per batch (Step Functions input limit)
  },
  "ResultWriter": {                 // write results to S3 instead of storing in parent execution
    "Resource": "arn:aws:states:::s3:putObject",
    "Parameters": {
      "Bucket": "my-tool-output-bucket",
      "Prefix.$": "States.Format('results/{}/', $$.Execution.Name)"
    }
  },
  "Iterator": {
    "StartAt": "ProcessBatch",
    "States": {
      "ProcessBatch": {
        "Type": "Task",
        "Resource": "arn:aws:lambda:us-east-1:123:function:process-tool-batch",
        "End": true
      }
    }
  }
};

MaxConcurrency: throttle external API calls to avoid rate limiting

MaxConcurrency: 0 means unbounded — Step Functions launches all iterations simultaneously regardless of how many items are in the array. For MCP tools that call external APIs, this is almost never correct. An agent sending 500 parallel tool calls to a third-party API that allows 100 requests per minute will produce 400 rate-limit errors, retry storms, and billing anomalies.

Set MaxConcurrency to a value that respects the rate limits of all downstream dependencies in the iterator. For Lambda tasks with no downstream rate limit, MaxConcurrency: 100 is a safe starting point — Lambda scales to 1,000 concurrent executions per region by default, and 100 concurrent Map iterations leaves headroom for other concurrent Map executions and direct Lambda invocations.

For tools that call external APIs, derive MaxConcurrency from the API's rate limit per second, divided by the expected per-iteration duration in seconds. Example: Stripe allows 100 requests per second; if each tool call iteration takes 0.5 seconds to call Stripe, max safe concurrency is 100 × 0.5 = 50 concurrent iterations.

import { SFNClient, DescribeMapRunCommand, ListMapRunsCommand } from '@aws-sdk/client-sfn';

const sfn = new SFNClient({ region: 'us-east-1' });

// Get the status of a Distributed Map run (not the parent execution)
// MapRuns are separate from executions for Distributed Map
async function describeMapRun(mapRunArn: string) {
  const run = await sfn.send(new DescribeMapRunCommand({ mapRunArn }));

  return {
    mapRunArn: run.mapRunArn,
    executionArn: run.executionArn, // parent execution ARN
    status: run.status, // RUNNING, SUCCEEDED, FAILED, ABORTED
    startDate: run.startDate,
    stopDate: run.stopDate,
    itemCounts: {
      pending: run.itemCounts?.pending,
      running: run.itemCounts?.running,
      succeeded: run.itemCounts?.succeeded,
      failed: run.itemCounts?.failed,
      timedOut: run.itemCounts?.timedOut,
      aborted: run.itemCounts?.aborted,
      total: run.itemCounts?.total,
      resultsWritten: run.itemCounts?.resultsWritten, // written to S3 ResultWriter
    },
    executionCounts: {
      pending: run.executionCounts?.pending,
      running: run.executionCounts?.running,
      succeeded: run.executionCounts?.succeeded,
      failed: run.executionCounts?.failed,
      total: run.executionCounts?.total,
    },
    toleratedFailure: {
      percentage: run.toleratedFailurePercentage,
      count: run.toleratedFailureCount,
    },
  };
}

// List all MapRuns for a parent execution — Distributed Map creates one MapRun per Map state visit
async function listMapRunsForExecution(executionArn: string) {
  const response = await sfn.send(new ListMapRunsCommand({ executionArn }));
  return response.mapRuns ?? [];
}

ItemBatcher: grouping items for batch-capable tool handlers

ItemBatcher is a Distributed Map-only feature that groups input items into sub-arrays before delivering them to the iterator. Without ItemBatcher, each Map iteration receives exactly one item. With ItemBatcher configured as MaxItemsPerBatch: 25, each iteration receives a batch of up to 25 items — allowing the iterator Lambda to call a batch API (DynamoDB BatchWriteItem, S3 bulk upload, or a tool handler that accepts arrays) instead of making one API call per item.

ItemBatcher introduces a subtle input shape change: the iterator no longer receives the item directly — it receives an object with a Items array containing the batch. Lambda handlers must be adapted accordingly.

// Lambda handler adapted for ItemBatcher input
// Without ItemBatcher: { item: {...}, index: 0, executionId: "..." }
// With ItemBatcher:    { Items: [{...}, {...}, ...], BatchInput: {...} }
export async function handler(event: {
  Items: Array<{ item: unknown; index: number; executionId: string }>;
  BatchInput?: unknown; // static input passed to all batches via ItemBatcher.BatchInput
}) {
  const { Items } = event;

  // Process all items in the batch
  const results = await Promise.allSettled(
    Items.map(({ item }) => processToolItem(item))
  );

  return results.map((result, i) => ({
    index: Items[i].index,
    status: result.status,
    value: result.status === 'fulfilled' ? result.value : null,
    reason: result.status === 'rejected' ? String(result.reason) : null,
  }));
}

Failure modes reference