Step Functions Distributed Map for Parallel MCP Tool Calls

Inline Map — small parallel batches

Inline Map processes items from the execution state input array. All results are collected into an array in the parent execution history. Use this for tool calls against up to 40 items where latency matters (no child execution overhead):

{
  "RunToolOnAllInputs": {
    "Type": "Map",
    "ItemsPath": "$.toolInputs",       // array in execution input
    "MaxConcurrency": 10,              // max 10 parallel iterations
    "ToleratedFailurePercentage": 20,  // fail Map state if >20% of items fail
    "Iterator": {
      "StartAt": "InvokeTool",
      "States": {
        "InvokeTool": {
          "Type": "Task",
          "Resource": "arn:aws:lambda:us-east-1:ACCOUNT:function:ToolHandler",
          "Parameters": {
            "toolName.$": "$.toolName",
            "input.$": "$.input",
            "index.$": "$$.Map.Item.Index"
          },
          "Retry": [
            {
              "ErrorEquals": ["Lambda.TooManyRequestsException"],
              "MaxAttempts": 3,
              "IntervalSeconds": 2,
              "BackoffRate": 1.5
            }
          ],
          "End": true
        }
      }
    },
    "ResultPath": "$.results",
    "End": true
  }
}

$$.Map.Item.Index provides the zero-based index of the current item within the Map iteration — useful for correlation when results are collected out of order. $$.Map.Item.Value provides the item value itself (same as $ within the iterator).

Distributed Map — large-scale parallel execution

Distributed mode creates independent child state machine executions. The input can be an inline array, an S3 JSON file, or an S3 NDJSON file. Results are written to S3 to avoid the 256 KB parent execution history limit:

{
  "ProcessLargeDataset": {
    "Type": "Map",
    "ItemReader": {
      "Resource": "arn:aws:states:::s3:getObject",
      "ReaderConfig": {
        "InputType": "JSON",
        "MaxItems": 10000
      },
      "Parameters": {
        "Bucket.$": "$.inputBucket",
        "Key.$": "$.inputKey"
      }
    },
    "ItemBatcher": {
      "MaxItemsPerBatch": 25,          // 25 items per child execution
      "MaxInputBytesPerBatch": 204800  // or 200 KB of input per batch, whichever is smaller
    },
    "MaxConcurrency": 50,
    "ToleratedFailurePercentage": 10,
    "ExecutionType": "STANDARD",
    "Label": "MCP-batch-tool-run",
    "Iterator": {
      "StartAt": "ProcessBatch",
      "States": {
        "ProcessBatch": {
          "Type": "Task",
          "Resource": "arn:aws:lambda:us-east-1:ACCOUNT:function:BatchToolHandler",
          "End": true
        }
      }
    },
    "ResultWriter": {
      "Resource": "arn:aws:states:::s3:putObject",
      "Parameters": {
        "Bucket.$": "$.outputBucket",
        "Prefix": "results/"
      }
    },
    "End": true
  }
}

With MaxItemsPerBatch: 25 and 10,000 total items, Distributed Map creates 400 child executions instead of 10,000. This reduces Lambda cold starts by 25× and stays well under the 300 child executions/second throttle. Each child execution's Lambda receives a batch of 25 items in event.Items.

S3ItemReader — processing datasets larger than 256 KB

The 256 KB execution input limit prevents passing large item arrays directly to the Map state. S3ItemReader streams items from S3 objects directly, bypassing the execution size limit:

# Supported input formats
# JSON array — each element becomes one Map item:
[
  { "url": "https://api.example.com/tool-1", "timeout": 5000 },
  { "url": "https://api.example.com/tool-2", "timeout": 5000 }
]

# NDJSON (newline-delimited JSON) — each line becomes one Map item:
{ "url": "https://api.example.com/tool-1", "timeout": 5000 }
{ "url": "https://api.example.com/tool-2", "timeout": 5000 }

# S3ItemReader config for NDJSON:
"ItemReader": {
  "Resource": "arn:aws:states:::s3:getObject",
  "ReaderConfig": {
    "InputType": "NDJSON"
    # MaxItems omitted = read all items
  },
  "Parameters": {
    "Bucket": "my-mcp-batch-inputs",
    "Key": "runs/2026-09-26/inputs.ndjson"
  }
}

S3ItemReader requires the state machine execution role to have s3:GetObject on the input bucket. The reader streams items through Step Functions — the entire file is never materialized in memory at once, so multi-GB input files are supported. Use NDJSON for large files: it can be produced incrementally (append-only) and parsed line by line without loading the entire structure.

ToleratedFailurePercentage and ToleratedFailureCount

By default, a single item failure causes the Map state to fail immediately (all in-flight iterations are cancelled). For MCP batch calls where partial success is acceptable, configure tolerance:

{
  "RunBatchToolCalls": {
    "Type": "Map",
    "ToleratedFailurePercentage": 15,   // fail Map only if >15% of items fail
    "ToleratedFailureCount": 100,       // fail Map if >100 items fail (absolute)
    // Both can be set — Map fails when EITHER threshold is crossed
    ...
  }
}

When the Map state completes with some failures but within tolerance, the ResultWriter output includes a manifest file (manifest.json) that lists which child executions succeeded and which failed, along with their execution ARNs. The parent state can read this manifest and decide whether to retry the failed subset:

# ResultWriter manifest structure (written to S3 Prefix/manifest.json)
{
  "DestinationBucket": "my-results-bucket",
  "DestinationPrefix": "results/",
  "ResultFiles": {
    "SUCCEEDED": [
      { "Key": "results/SUCCESS_0.json", "ItemCount": 25 },
      { "Key": "results/SUCCESS_1.json", "ItemCount": 25 }
    ],
    "FAILED": [
      { "Key": "results/FAILED_0.json", "ItemCount": 3 }
    ]
  }
}

ItemBatcher — reducing child execution overhead

Without ItemBatcher, each item in the input creates one child execution or one Map iteration. For 10,000 items with a Lambda that takes 2 seconds per item, that is 10,000 Lambda cold starts (unless Lambda concurrency keeps containers warm). ItemBatcher groups items into batches and passes each batch to one Lambda invocation:

# Lambda receives batched items in event.Items array
export async function handler(event: {
  Items: Array<{ url: string; timeout: number }>
}): Promise<{ results: Array<{ url: string; status: number }> }> {
  // Process all items in the batch
  const results = await Promise.all(
    event.Items.map(item =>
      probe(item.url, item.timeout).catch(err => ({ url: item.url, status: 0, error: err.message }))
    )
  );
  return { results };
}

# ItemBatcher config — groups up to 25 items or 200 KB per batch, whichever limit hits first
"ItemBatcher": {
  "MaxItemsPerBatch": 25,
  "MaxInputBytesPerBatch": 204800,
  "BatchInput": {            // additional static fields merged into every batch's input
    "environment": "production",
    "toolVersion": "2"
  }
}

BatchInput merges static fields into every batch input alongside the Items array — useful for passing environment-specific configuration without including it in each item. The Lambda function receives both event.Items (the batch) and event.environment, event.toolVersion (from BatchInput).