AWS Textract · 2026-11-03 · Textract Advanced arc
AWS Textract Document Intelligence for MCP Servers: Block Model, Queries, Table Extraction, Expense Parsing, and Async Jobs
AliveMCP monitors MCP endpoints every 60 seconds — but the endpoints that fail most silently are document-processing tools built on AWS Textract, because Textract's failure modes do not produce exceptions: the sync API quietly drops pages 2 through N of a multi-page PDF with no error, the Queries API returns an empty result for a misphased question without throwing, a FORMS extractor returns 60 KEY_VALUE_SET blocks when you only needed two fields, and a table extractor that ignores RowSpan produces a grid where every row after a merged cell is shifted one column off. Four Textract capabilities close the gap between a prototype document tool and a production-grade document intelligence MCP server. The Block model and core APIs are the foundation: every Textract response — regardless of which API you call — returns a flat array of Block objects, each identified by a UUID in its Id field, classified by BlockType (PAGE, LINE, WORD, TABLE, CELL, KEY_VALUE_SET, QUERY, QUERY_RESULT, SIGNATURE, SELECTION_ELEMENT), scored with a Confidence float from 0–100, placed in the document with a Geometry object containing bounding box and polygon, and linked to related blocks via a Relationships array of typed ID lists — navigation is entirely through these relationships, there is no nested tree; the correct pattern is to build an id→block Map once from the full blocks array and then walk relationships; the four sync APIs each expose different block types: DetectDocumentText returns PAGE/LINE/WORD blocks only (raw text extraction, accepts inline bytes up to 10 MB or S3 reference, one page limit for PDFs, use LINE blocks filtered from the response for full text); AnalyzeDocument adds KEY_VALUE_SET blocks (FORMS feature), TABLE/CELL blocks (TABLES feature), QUERY/QUERY_RESULT blocks (QUERIES feature), and SIGNATURE blocks — FeatureTypes controls which are added, you pay per feature type used, and passing all four when you only need tables adds latency and cost without benefit; AnalyzeExpense returns ExpenseDocuments each containing SummaryFields (vendor name, invoice date, total, tax, payment terms, PO number — semantically normalized across supplier template variations so "Invoice No." and "Ref. #" both map to INVOICE_RECEIPT_ID) and LineItemGroups (ITEM, QUANTITY, UNIT_PRICE, PRICE per row, with EXPENSE_ROW as a fallback full-row text string when column parsing fails); AnalyzeID uses DocumentPages (an array, not Document singular) and returns IdentityDocumentFields with typed fields like FIRST_NAME, DATE_OF_BIRTH, EXPIRATION_DATE. The Queries API is the right architecture for targeted field extraction when document template variation makes FORMS extraction brittle: instead of scanning 60 KEY_VALUE_SET blocks for the right key string, you ask up to 15 natural-language questions via QueriesConfig (each query has a Text string, an optional Alias for stable output keys, and optional Pages filter for multi-page async documents); Textract searches the entire document including running prose and tables — not just structured form regions — and returns QUERY blocks each optionally linked to a QUERY_RESULT block via a Relationships[{Type:'ANSWER'}] entry; a missing ANSWER relationship means Textract found no plausible answer location (different from a low-confidence result — check both cases); confidence interpretation: 90–100 use directly, 70–89 use and log for spot-check, 50–69 flag for human review, below 50 treat as not found; the parameterized MCP tool accepts an S3 URI and a fields array of alias/question/pages objects, letting an AI orchestrator dynamically specify what to extract from unknown document types without modifying the MCP server. Table extraction requires graph traversal of the TABLE→CELL→WORD block graph: each TABLE block has CHILD relationships to CELL blocks; each CELL block has RowIndex and ColumnIndex (both 1-based), RowSpan and ColumnSpan (both default 1 for normal cells), EntityTypes (COLUMN_HEADER, MERGED_CELL, TABLE_TITLE, TABLE_FOOTER, TABLE_SECTION_TITLE, SEMI_STRUCTURED_TABLE), and CHILD relationships to WORD blocks or a SELECTION_ELEMENT block for checkbox cells; the grid dimensions are not directly available — you compute them as max(RowIndex + RowSpan - 1) and max(ColumnIndex + ColumnSpan - 1) across all cells; merged cells are the most common source of incorrect extraction: a merged cell has one CELL block with RowSpan or ColumnSpan greater than 1, and only covers its primary position — the spanned positions have no blocks at all, so a naive grid-placement loop that ignores RowSpan will shift all subsequent cells one position off in the affected rows; the fix is to explicitly fill all spanned grid positions with the merged cell's text before moving to the next cell; TABLE_TITLE and TABLE_FOOTER relationship types on TABLE blocks require casting r.Type to string in older SDK TypeScript types that don't include them as valid enum values. Async document analysis is not optional for multi-page PDFs — the sync APIs handle only one page and silently drop the rest, which is the single most common Textract integration mistake; StartDocumentAnalysisCommand accepts S3-only input (inline bytes are not supported), processes up to 3,000 pages, and returns a JobId immediately; the NotificationChannel requires a dedicated IAM role that Textract assumes (not the caller's role) to publish to SNS — trust policy must grant sts:AssumeRole to textract.amazonaws.com with an aws:SourceAccount condition to prevent confused-deputy attacks, and the role's permissions policy must grant sns:Publish on the topic; the calling principal additionally needs iam:PassRole on this notification role ARN; GetDocumentAnalysisCommand with MaxResults: 1 is the lightweight status-check pattern — it retrieves JobStatus without transferring the full block list; pagination with MaxResults: 1000 and NextToken loops must complete and all block pages assembled into a single array before any relationship traversal — TABLE blocks reference CELL blocks by ID, and those cells may be in a different response page; the three-tool MCP pattern exposes async jobs without blocking: start_document_analysis submits and returns jobId, check_analysis_status polls status, and get_analysis_results paginates and extracts — all three handlers return in under one second except the final retrieve. Combined with AnalyzeExpense for financial document processing, these four Textract patterns cover the full spectrum of document intelligence use cases an AI agent needs: raw text extraction, targeted field queries, structured table data, financial document parsing, and large-document async processing. This guide synthesises all four. Pattern 1 covers the Block model — shared data structure, API selection, id-index navigation, IAM permissions, and four MCP tool patterns — from AWS Textract in MCP Servers. Pattern 2 covers the Queries API — QUERY/QUERY_RESULT block navigation, confidence thresholds, page targeting, combining with TABLES, and parameterized MCP tool schema — from Textract Queries API in MCP Servers. Pattern 3 covers table extraction — TABLE→CELL graph traversal, grid assembly, merged cells, header detection, and serialization — from Textract Table Extraction in MCP Servers. Pattern 4 covers expense parsing and async jobs — AnalyzeExpense SummaryFields and LineItemGroups, AP automation tool pattern, StartDocumentAnalysis with SNS, GetDocumentAnalysis pagination, and the three-tool async MCP pattern — from Textract AnalyzeExpense in MCP Servers and Textract Async Document Analysis in MCP Servers.
TL;DR
- Block model — build
id→BlockMap once, walk Relationships to navigate; never trust block array order; sync APIs process 1 page for PDFs (silent truncation, not an error): Index all blocks:const idx = new Map(blocks.map(b => [b.Id!, b])). Get children: filterblock.RelationshipsforType === 'CHILD', then look up each ID. For FORMS KEY_VALUE_SET blocks, KEY→VALUE viaType === 'VALUE'relationship; VALUE's CHILD blocks are WORD (text) or SELECTION_ELEMENT (checkbox). Choose the right API: DetectDocumentText for raw text; AnalyzeDocument for forms/tables/queries/signatures; AnalyzeExpense for invoices; AnalyzeID for identity documents. Single page only for all sync APIs on PDFs — use StartDocumentAnalysis for any document over one page. IAM minimum:textract:DetectDocumentText,textract:AnalyzeDocument,textract:AnalyzeExpense,textract:AnalyzeID,s3:GetObjecton your document bucket. Textract has no VPC endpoint — Lambda/ECS tasks need internet access for the Textract API call. - Queries API — ask up to 15 natural-language questions per call; match QUERY blocks to results by Alias not array position; missing ANSWER relationship means "not found", not "low confidence": Always pair
QueriesConfigwithFeatureTypes: [FeatureType.QUERIES]— omitting either one silently fails. UseQuery.Aliasas stable output keys: iterateblocks.filter(b => b.BlockType === 'QUERY'), readblock.Query.Alias, find ANSWER relationship to get result block ID, readresultBlock.TextandresultBlock.Confidence. When ANSWER relationship is absent, the field was not found — returnnull, not empty string. Confidence below 50 = treat as not found. Queries beats FORMS when: document templates vary across suppliers, target values may appear in prose (not labeled input fields), or you want runtime-parameterized extraction. Page targeting in async path:Pages: ['1']restricts search scope and improves accuracy for known field positions.z.array(...).max(15)in your MCP tool schema — Textract rejects more than 15 queries per call. - Table extraction — grid dimensions require computing
max(RowIndex+RowSpan-1); merged cells must fill all spanned positions or downstream rows misalign; always assemble all async block pages before traversal: Build grid:Array.from({length:maxRow}, () => Array(maxCol).fill('')). For each CELL:row = RowIndex-1, col = ColumnIndex-1. Fill merged cells: double loop overRowSpan × ColumnSpan— not just the primary cell. Detect headers:(cell.EntityTypes ?? []).includes('COLUMN_HEADER'); fallback strategy: treat row 0 as header unconditionally for well-structured PDFs. Handle checkboxes: child SELECTION_ELEMENT withSelectionStatus === 'SELECTED'→'TRUE'. Castr.Type as stringwhen checking TABLE_TITLE/TABLE_FOOTER relationships (not valid enum values in older SDK). Serialize: JSON array of objects for orchestrator reasoning, Markdown table for LLM context, CSV for downstream data tools. Multi-page: group TABLE blocks byblock.Pagefor page-specific processing in async results. - Async jobs + Expense — sync AnalyzeDocument silently drops pages 2+ of multi-page PDFs; StartDocumentAnalysis needs a dedicated Textract-assume SNS role; assemble all GetDocumentAnalysis pages before block traversal: Async:
StartDocumentAnalysisCommandS3 only, up to 3,000 pages. Notification role trust:Principal: {Service: 'textract.amazonaws.com'}+Condition: {StringEquals: {'aws:SourceAccount': 'YOUR_ACCOUNT_ID'}}. Caller needsiam:PassRoleon the notification role ARN. Lightweight status:GetDocumentAnalysis({JobId, MaxResults:1}). Full retrieval: loopGetDocumentAnalysis({JobId, MaxResults:1000, NextToken})until no NextToken; collect all blocks first, then index and traverse. Three-tool MCP: start (returns jobId) → check (returns status) → retrieve (returns extracted data). Expense:AnalyzeExpenseCommandreturnsExpenseDocuments[]withSummaryFields(VENDOR_NAME, INVOICE_RECEIPT_ID, INVOICE_RECEIPT_DATE, DUE_DATE, PAYMENT_TERMS, SUBTOTAL, TAX, TOTAL, PO_NUMBER) andLineItemGroups(ITEM, QUANTITY, UNIT_PRICE, PRICE, EXPENSE_ROW). Set confidence floor (60) and returnnullfor fields below it. AP tool pattern:analyze_invoice→validate_invoice→post_invoice.
Pattern 1: The Block Model — the universal data structure underlying all Textract APIs
The single most important thing to understand about AWS Textract is that all four of its primary APIs — DetectDocumentText, AnalyzeDocument, AnalyzeExpense, and AnalyzeID — return data through the same underlying model: a flat array of Block objects. There is no nested tree returned directly. Paragraph text does not contain word objects as children; tables do not nest cell objects directly. Everything is in a single flat array, and the structure is expressed entirely through Relationships arrays that link block IDs together.
Block types and what they represent
The most common BlockType values an MCP server tool handler will encounter:
- PAGE — one block per document page; parent of all LINE blocks on that page via CHILD relationships
- LINE — a detected line of text; contains WORD children; the most useful block for assembling plain document text
- WORD — a single word with a Confidence score (0–100) and bounding box Geometry
- TABLE — a detected tabular structure; CHILD relationships point to CELL blocks
- CELL — one cell in a table; has RowIndex, ColumnIndex, RowSpan, ColumnSpan, and EntityTypes; CHILD relationships point to WORD or SELECTION_ELEMENT blocks
- KEY_VALUE_SET — a form field key or value; EntityTypes distinguishes KEY from VALUE; KEY blocks have a VALUE relationship pointing to the VALUE block
- SELECTION_ELEMENT — a checkbox or radio button; SelectionStatus is SELECTED or NOT_SELECTED
- QUERY — the natural-language question you submitted; may have an ANSWER relationship pointing to a QUERY_RESULT block
- QUERY_RESULT — the answer Textract found; has Text and Confidence
- SIGNATURE — a detected signature region; added when FeatureType.SIGNATURES is requested
The id-index navigation pattern
Because the response is a flat array, the first operation in every Textract tool handler should be building an id→Block Map. Never rely on array position or assume parent and child blocks are adjacent:
import { type Block } from '@aws-sdk/client-textract';
function buildIndex(blocks: Block[]): Map<string, Block> {
const map = new Map<string, Block>();
for (const block of blocks) {
if (block.Id) map.set(block.Id, block);
}
return map;
}
function getRelated(block: Block, type: string, idx: Map<string, Block>): Block[] {
return (block.Relationships ?? [])
.filter(r => r.Type === type)
.flatMap(r => r.Ids ?? [])
.map(id => idx.get(id))
.filter((b): b is Block => b !== undefined);
}
// Extract text from any block that has WORD children
function blockText(block: Block, idx: Map<string, Block>): string {
if (block.BlockType === 'WORD') return block.Text ?? '';
return getRelated(block, 'CHILD', idx)
.filter(b => b.BlockType === 'WORD')
.map(b => b.Text ?? '')
.join(' ');
}
Choosing the right API for each document type
The four sync Textract APIs are not interchangeable — each has a different response structure and is optimized for a different document category:
| Document type | API | Key response structure | MCP tool use case |
|---|---|---|---|
| Scanned text, contracts, reports | DetectDocumentText | LINE + WORD blocks | Extract → summarize → Q&A |
| Structured forms, checkboxes | AnalyzeDocument (FORMS) | KEY_VALUE_SET blocks | Form field extraction |
| Documents with data tables | AnalyzeDocument (TABLES) | TABLE + CELL blocks | Financial table extraction, CSV output |
| Any doc — targeted fields | AnalyzeDocument (QUERIES) | QUERY + QUERY_RESULT blocks | Runtime-parameterized extraction |
| Invoices, receipts, credit notes | AnalyzeExpense | ExpenseDocuments with SummaryFields + LineItemGroups | AP automation, expense reports |
| Driver's licenses, passports | AnalyzeID | IdentityDocumentFields | KYC, identity verification |
| Multi-page PDFs (>1 page) | StartDocumentAnalysis (async) | Same block types via paginated GetDocumentAnalysis | Any sync use case at scale |
The single hardest-to-find bug in Textract integrations is calling a sync API on a multi-page PDF. DetectDocumentText and AnalyzeDocument accept PDFs in their Document.Bytes or Document.S3Object parameters, and they process the input successfully — but they only process page 1. No exception is thrown, no warning is logged, no truncation signal appears in the response. The result simply contains blocks from the first page only. If your invoice or contract has the total on page 2 and you call sync AnalyzeDocument, you will get a clean response with zero results for any query targeting that page. See the async guide for the correct pattern when processing multi-page documents.
IAM and networking requirements
The minimum IAM policy for an MCP server using all four sync Textract APIs:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"textract:DetectDocumentText",
"textract:AnalyzeDocument",
"textract:AnalyzeExpense",
"textract:AnalyzeID"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": ["s3:GetObject"],
"Resource": "arn:aws:s3:::your-document-bucket/*"
}
]
}
A hard networking constraint: Textract has no VPC endpoint for its data-plane APIs. Lambda functions or ECS tasks running Textract MCP tools need internet access to reach the public Textract HTTPS endpoint. Outbound internet access via a NAT Gateway (for private subnets) or direct internet gateway (for public subnets) is required. S3 access can go through a VPC gateway endpoint at no cost, but the Textract API call itself must route through the internet or a VPC endpoint for Interface Endpoint (check current availability by region).
Pattern 2: The Queries API — targeted field extraction with natural-language questions
The FORMS feature type in AnalyzeDocument extracts every key-value pair on the page and returns them as KEY_VALUE_SET blocks. For an invoice with 30 labeled fields, you get 60 blocks (30 keys, 30 values). You then search for the specific key text you care about — which requires fuzzy matching because "Invoice Total", "Total Due", and "TOTAL" are all different strings for the same concept. The visual proximity heuristic that drives FORMS extraction also breaks on non-standard layouts where label and input are not in classic side-by-side or above-below positions.
The Queries API sidesteps all of this. You ask a natural-language question, and Textract's model searches the entire document — including prose paragraphs, tables, and standalone values — to find an answer. A query like "What is the total amount due?" can find the answer in running text ("The total amount due is $1,542.00") that FORMS would miss entirely because there is no structured key-value layout to detect.
QUERY and QUERY_RESULT blocks
Each submitted query generates exactly one QUERY block. If Textract finds an answer, the QUERY block has a Relationships entry with Type: 'ANSWER' pointing to one QUERY_RESULT block. If no answer is found, the QUERY block exists but has no ANSWER relationship at all — the QUERY_RESULT block simply does not exist.
This distinction matters for production code: a QUERY block with no ANSWER relationship means Textract found no plausible location for the answer anywhere in the document. A low-confidence QUERY_RESULT means Textract found something but is uncertain about it. Both cases should return null in your MCP tool output — the orchestrator should not interpret low-confidence extractions as reliable data, and absent fields should not produce empty strings that downstream tools interpret as found-but-blank.
import {
TextractClient,
AnalyzeDocumentCommand,
FeatureType,
type Block,
type QueriesConfig,
} from '@aws-sdk/client-textract';
const client = new TextractClient({ region: 'us-east-1' });
interface QueryResult {
alias: string;
answer: string | null;
confidence: number | null;
page: number | null;
}
async function extractWithQueries(
bucket: string,
key: string,
queries: Array<{ text: string; alias: string; pages?: string[] }>,
): Promise<QueryResult[]> {
const resp = await client.send(new AnalyzeDocumentCommand({
Document: { S3Object: { Bucket: bucket, Name: key } },
FeatureTypes: [FeatureType.QUERIES], // REQUIRED — omit and QueriesConfig is ignored
QueriesConfig: {
Queries: queries.map(q => ({
Text: q.text,
Alias: q.alias,
Pages: q.pages,
})),
},
}));
const blocks = resp.Blocks ?? [];
const idx = new Map<string, Block>(
blocks.filter(b => b.Id).map(b => [b.Id!, b])
);
return blocks
.filter(b => b.BlockType === 'QUERY')
.map(b => {
const alias = b.Query?.Alias ?? b.Query?.Text ?? '';
const answerRel = (b.Relationships ?? []).find(r => r.Type === 'ANSWER');
if (!answerRel?.Ids?.length) {
return { alias, answer: null, confidence: null, page: null };
}
const result = idx.get(answerRel.Ids[0]);
return {
alias,
answer: result?.Text ?? null,
confidence: result?.Confidence ?? null,
page: result?.Page ?? null,
};
});
}
Confidence thresholds for production pipelines
Confidence scores from QUERY_RESULT blocks reflect how certain Textract is about both the location and accuracy of the answer. Empirical thresholds for financial document processing:
| Confidence range | Interpretation | Recommended handling |
|---|---|---|
| 90–100 | High — exact match found | Use directly in downstream systems |
| 70–89 | Good — likely correct | Use; log for spot-check auditing |
| 50–69 | Moderate — possible but ambiguous | Flag for human review before posting |
| Below 50 | Low — field may not exist or layout non-standard | Treat as not found; return null |
| No QUERY_RESULT block | Textract found no candidate answer location | Field absent; do not infer or default |
Parameterized MCP tool for dynamic document extraction
The Queries API enables a generic extraction tool where the AI orchestrator specifies what to extract at runtime. The MCP server does not need modification when a new document type is introduced — the orchestrator supplies the field definitions:
// MCP tool: parameterized document field extractor
server.tool(
'extract_document_fields',
{
s3_uri: z.string().describe('S3 URI: s3://bucket/key'),
fields: z.array(z.object({
alias: z.string().describe('Stable output key'),
question: z.string().describe('Natural-language question'),
pages: z.array(z.string()).optional().describe('Page filters like ["1"] or ["2-4"]'),
})).max(15).describe('Up to 15 field definitions'),
confidence_threshold: z.number().min(0).max(100).default(60),
},
async ({ s3_uri, fields, confidence_threshold }) => {
const url = new URL(s3_uri);
const results = await extractWithQueries(
url.hostname,
url.pathname.slice(1),
fields.map(f => ({ text: f.question, alias: f.alias, pages: f.pages })),
);
const extracted: Record<string, unknown> = {};
const missing: string[] = [];
for (const r of results) {
if (r.answer !== null && (r.confidence ?? 0) >= confidence_threshold) {
extracted[r.alias] = { value: r.answer, confidence: r.confidence, page: r.page };
} else {
missing.push(r.alias);
}
}
return { content: [{ type: 'text', text: JSON.stringify({ extracted, missing }, null, 2) }] };
},
);
The missing array gives the orchestrator a clean signal for which fields were not found. The orchestrator can decide independently whether to route to human review, skip downstream steps that require those fields, or retry with rephrased questions — without any changes to the MCP server. For multi-page documents in the async path, Pages: ['1'] restricts a query to a specific page, improving accuracy when you know which page of a contract contains the target field. See the Queries API guide for StartDocumentAnalysis with QueriesConfig for the async path.
Pattern 3: Table Extraction — CELL graph traversal and merged-cell handling
Textract's TABLES feature detects tabular structures in documents regardless of whether they use visible grid lines, color banding, or borderless whitespace alignment. The response expresses tables as a graph of TABLE→CELL→(WORD|SELECTION_ELEMENT) blocks connected via CHILD relationships. Reconstructing the 2D grid from this graph is straightforward for simple tables, but two sources of complexity — merged cells and multi-row headers — account for the majority of production table extraction bugs.
Grid assembly from RowIndex and ColumnIndex
Each CELL block has RowIndex and ColumnIndex (both 1-based) and RowSpan and ColumnSpan (both default to 1 for normal cells). There is no explicit row count or column count on TABLE blocks — you derive them by computing the maximum extent across all cells:
type Grid = string[][];
function extractTable(table: Block, idx: Map<string, Block>): Grid {
// Get all CELL blocks for this table
const cells = (table.Relationships ?? [])
.filter(r => r.Type === 'CHILD')
.flatMap(r => r.Ids ?? [])
.map(id => idx.get(id))
.filter((b): b is Block => b !== undefined && b.BlockType === 'CELL');
// Derive grid dimensions from cell extent (not a direct TABLE field)
const maxRow = Math.max(...cells.map(c => (c.RowIndex ?? 1) + (c.RowSpan ?? 1) - 1));
const maxCol = Math.max(...cells.map(c => (c.ColumnIndex ?? 1) + (c.ColumnSpan ?? 1) - 1));
const grid: Grid = Array.from({ length: maxRow }, () => Array(maxCol).fill(''));
for (const cell of cells) {
const row = (cell.RowIndex ?? 1) - 1; // convert 1-based to 0-based
const col = (cell.ColumnIndex ?? 1) - 1;
// Get cell text (or checkbox state for SELECTION_ELEMENT cells)
const children = (cell.Relationships ?? [])
.filter(r => r.Type === 'CHILD')
.flatMap(r => r.Ids ?? [])
.map(id => idx.get(id))
.filter((b): b is Block => b !== undefined);
const selEl = children.find(b => b.BlockType === 'SELECTION_ELEMENT');
const text = selEl
? (selEl.SelectionStatus === 'SELECTED' ? 'TRUE' : 'FALSE')
: children.filter(b => b.BlockType === 'WORD').map(b => b.Text ?? '').join(' ');
// Fill primary position
grid[row][col] = text;
// Fill all spanned positions for merged cells — CRITICAL
const rSpan = cell.RowSpan ?? 1;
const cSpan = cell.ColumnSpan ?? 1;
for (let r = row; r < row + rSpan; r++) {
for (let c = col; c < col + cSpan; c++) {
if (r !== row || c !== col) grid[r][c] = text;
}
}
}
return grid;
}
Why merged cells break grid extraction
When a table has a merged cell spanning rows 1–2 in column 1, Textract returns exactly one CELL block for that merge region — positioned at RowIndex=1, ColumnIndex=1, RowSpan=2. There is no placeholder block for row 2, column 1. If you place cells naively by RowIndex/ColumnIndex without accounting for span, the cells in rows below the merged cell get placed at the wrong column offset because the grid positions they should occupy are empty.
The fix is the double loop above: for every cell with RowSpan or ColumnSpan greater than 1, explicitly write the cell's text into every grid position it covers — not just the primary (RowIndex, ColumnIndex) position. The alternative, for tools that need accurate "each physical cell has one value" semantics (like CSV exporters), is to leave spanned positions empty and only fill the primary position. Choose based on downstream use.
Column header detection and multi-row headers
Textract marks cells it classifies as column headers with EntityTypes: ['COLUMN_HEADER']. This detection is reliable for documents with standard visual header formatting (bold, top-row position, background shading). For non-standard documents, a robust fallback is to treat row index 0 as the header unconditionally:
function gridToObjects(grid: Grid, idx: Map<string, Block>, cells: Block[]): Array<Record<string, string>> {
// Detect which rows Textract classified as headers
const headerRowSet = new Set(
cells
.filter(c => (c.EntityTypes ?? []).includes('COLUMN_HEADER'))
.map(c => (c.RowIndex ?? 1) - 1)
);
// Fall back to row 0 if no COLUMN_HEADER entities detected
const headerRows = headerRowSet.size > 0
? grid.filter((_, i) => headerRowSet.has(i))
: [grid[0]];
const dataRows = headerRowSet.size > 0
? grid.filter((_, i) => !headerRowSet.has(i))
: grid.slice(1);
// Use last header row as column names (for multi-row headers, last row has sub-labels)
const headers = headerRows[headerRows.length - 1] ?? [];
return dataRows.map(row =>
Object.fromEntries(headers.map((h, i) => [h || `col_${i}`, row[i] ?? '']))
);
}
For multi-row headers (a top "group" row spanning multiple columns, with specific sub-labels below), combine both rows into compound keys: `${groupLabel} — ${subLabel}`. This produces headers like "Q1 — Revenue" and "Q1 — Units" from a quarterly table, preserving group membership in the column names.
Serialization for MCP tool responses
MCP tools return text content. The right serialization format depends on how the AI orchestrator will use the table:
function tableToMarkdown(headers: string[], rows: string[][]): string {
const sep = `| ${headers.map(() => '---').join(' | ')} |`;
return [
`| ${headers.join(' | ')} |`,
sep,
...rows.map(r => `| ${r.join(' | ')} |`),
].join('\n');
}
function tableToCsv(headers: string[], rows: string[][]): string {
const esc = (s: string) =>
/[,"\n]/.test(s) ? `"${s.replace(/"/g, '""')}"` : s;
return [headers, ...rows].map(r => r.map(esc).join(',')).join('\n');
}
Use JSON array of objects (from gridToObjects) when the orchestrator will reason about specific column values or pass data to a downstream tool. Use Markdown for passing tables to a language model for reasoning or summarization. Use CSV for code-execution environments or data import tools. See the full table extraction guide for the complete TABLE_TITLE/TABLE_FOOTER relationship extraction pattern and multi-page table grouping by block.Page.
Pattern 4: Expense Analysis and Async Jobs — financial document parsing and multi-page processing
Two Textract capabilities address the two highest-friction document processing scenarios: AnalyzeExpense handles the complexity of financial document variation (different supplier templates, different label conventions, different currency formats), and the async API handles the volume constraint that makes sync APIs unsuitable for anything larger than a single page.
AnalyzeExpense: semantic normalization over FORMS
The key advantage of AnalyzeExpense over AnalyzeDocument with FORMS is semantic normalization. The FORMS feature returns raw label text as the key — "Invoice No.", "Invoice #", "Bill Number", "Ref. #" each become different keys for the same field. AnalyzeExpense normalizes all of these to INVOICE_RECEIPT_ID. Your code checks field.Type.Text === 'INVOICE_RECEIPT_ID' regardless of which supplier issued the invoice.
The response structure has two levels: SummaryFields for document-level metadata and LineItemGroups for per-item rows:
import {
TextractClient,
AnalyzeExpenseCommand,
type ExpenseField,
} from '@aws-sdk/client-textract';
const client = new TextractClient({ region: 'us-east-1' });
const CONFIDENCE_FLOOR = 60;
function getFieldValue(field: ExpenseField): { type: string; value: string } | null {
const type = field.Type?.Text;
const value = field.ValueDetection?.Text;
const confidence = field.ValueDetection?.Confidence ?? 0;
if (!type || !value || confidence < CONFIDENCE_FLOOR) return null;
return { type, value };
}
async function parseInvoice(bucket: string, key: string) {
const resp = await client.send(new AnalyzeExpenseCommand({
Document: { S3Object: { Bucket: bucket, Name: key } },
}));
const doc = resp.ExpenseDocuments?.[0];
if (!doc) return null;
// Summary fields: VENDOR_NAME, INVOICE_RECEIPT_ID, INVOICE_RECEIPT_DATE,
// DUE_DATE, PAYMENT_TERMS, SUBTOTAL, TAX, TOTAL, PO_NUMBER, RECEIVER_NAME
const summary: Record<string, string[]> = {};
for (const field of doc.SummaryFields ?? []) {
const extracted = getFieldValue(field);
if (!extracted || extracted.type === 'OTHER') continue;
(summary[extracted.type] ??= []).push(extracted.value);
}
// Multiple entries for the same type are joined with ' / '
const get = (type: string) => summary[type]?.join(' / ') ?? null;
// Line items: ITEM, QUANTITY, UNIT_PRICE, PRICE; EXPENSE_ROW as fallback
const lineItems = (doc.LineItemGroups ?? []).flatMap(g =>
(g.LineItems ?? []).map(item => {
const fields: Record<string, string> = {};
for (const f of item.LineItemExpenseFields ?? []) {
const e = getFieldValue(f);
if (e) fields[e.type] = e.value;
}
return {
description: fields['ITEM'] ?? fields['EXPENSE_ROW'] ?? '',
quantity: fields['QUANTITY'] ?? '',
unitPrice: fields['UNIT_PRICE'] ?? '',
amount: fields['PRICE'] ?? fields['EXPENSE_ROW'] ?? '',
};
})
);
return {
vendorName: get('VENDOR_NAME'),
invoiceNumber: get('INVOICE_RECEIPT_ID'),
invoiceDate: get('INVOICE_RECEIPT_DATE'),
dueDate: get('DUE_DATE'),
paymentTerms: get('PAYMENT_TERMS'),
total: get('TOTAL'),
tax: get('TAX'),
subtotal: get('SUBTOTAL'),
poNumber: get('PO_NUMBER'),
lineItems,
};
}
EXPENSE_ROW is Textract's fallback for line items where individual column parsing failed — it contains the full row text concatenated. When ITEM, QUANTITY, and PRICE are all absent for a line item, check EXPENSE_ROW and either parse it with a regex or pass it to a language model. Currency amounts are returned as raw document strings ("$1,512.00", "1.512,00" in European format) — parse them to numeric values in your tool rather than passing raw strings to accounting systems. See the AnalyzeExpense guide for the full currency parsing implementation and multi-document PDF handling via ExpenseDocuments[].ExpenseIndex.
AP automation tool pattern
The right architecture for accounts payable automation separates extraction, validation, and action into three distinct MCP tools that the AI orchestrator calls in sequence:
| Tool | Input | Operation | Output |
|---|---|---|---|
analyze_invoice |
S3 URI | AnalyzeExpense → structured object | ParsedInvoice JSON |
validate_invoice |
ParsedInvoice | Rules: required fields, line item sum = subtotal, subtotal + tax = total, vendor in master | ValidationReport: pass | exceptions[] |
post_invoice |
ParsedInvoice + GL mapping | Post to accounting API (QuickBooks, NetSuite) | Transaction ID or error |
The orchestrator routes analyze_invoice → validate_invoice → human approval on exceptions → post_invoice. Separating extraction into its own tool makes it easy to substitute a different extraction provider (e.g., Azure Document Intelligence) without modifying the validation and posting tools.
Async jobs: StartDocumentAnalysis and SNS notification
For any PDF over one page, the async path is mandatory. Calling sync AnalyzeDocument on a multi-page PDF returns clean results for page 1 only — no exception, no warning, no truncation signal. This silent behavior is the single most common Textract production bug.
The NotificationChannel parameter requires a dedicated IAM role — not the caller's role — that Textract assumes when the job completes to publish to SNS. The trust policy requires an aws:SourceAccount condition to prevent confused-deputy attacks where another AWS customer's Textract job could use your notification role:
# Trust policy (allows Textract service principal to assume this role)
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Principal": { "Service": "textract.amazonaws.com" },
"Action": "sts:AssumeRole",
"Condition": {
"StringEquals": { "aws:SourceAccount": "YOUR_ACCOUNT_ID" }
}
}]
}
# Permissions policy (SNS publish only)
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Action": "sns:Publish",
"Resource": "arn:aws:sns:us-east-1:YOUR_ACCOUNT_ID:textract-completion"
}]
}
The calling principal (your Lambda or ECS task role) needs textract:StartDocumentAnalysis, textract:GetDocumentAnalysis, s3:GetObject on the source document, and critically iam:PassRole on the notification role ARN — without PassRole, the StartDocumentAnalysis call is rejected with an error even if the notification role itself is correctly configured.
Paginating GetDocumentAnalysis correctly
Each GetDocumentAnalysis response page contains up to 1,000 blocks and may include a NextToken for the next page. The critical constraint: do not traverse block relationships until you have assembled all pages. TABLE blocks reference CELL blocks by ID, and those CELL blocks can be in a different response page than the TABLE block. If you process pages one at a time and try to resolve relationships on each page, you will encounter missing IDs and produce empty or partial tables:
async function getAllBlocks(client: TextractClient, jobId: string): Promise<Block[]> {
const all: Block[] = [];
let nextToken: string | undefined;
do {
const resp = await client.send(new GetDocumentAnalysisCommand({
JobId: jobId,
MaxResults: 1000,
NextToken: nextToken,
}));
if (resp.JobStatus === 'FAILED') {
throw new Error(`Job failed: ${resp.StatusMessage ?? 'unknown'}`);
}
all.push(...(resp.Blocks ?? []));
nextToken = resp.NextToken;
} while (nextToken !== undefined);
return all; // Build the id→Block index from all here, then traverse
}
// Lightweight status check (no block data transferred)
async function getJobStatus(client: TextractClient, jobId: string) {
const resp = await client.send(new GetDocumentAnalysisCommand({
JobId: jobId,
MaxResults: 1, // Minimizes data transfer — status is in every response
}));
return { status: resp.JobStatus, message: resp.StatusMessage };
}
Three-tool MCP pattern for non-blocking async jobs
MCP tool handler timeouts are typically 30–60 seconds. A Textract job on a 100-page document can take several minutes. Blocking a tool handler to wait for completion is not viable. The three-tool pattern exposes async jobs to orchestrators without any blocking:
// Tool 1: submit (returns immediately with jobId)
server.tool('start_document_analysis', {
s3_uri: z.string(),
features: z.array(z.enum(['TABLES', 'FORMS', 'QUERIES', 'SIGNATURES'])),
}, async ({ s3_uri, features }) => {
const url = new URL(s3_uri);
const resp = await client.send(new StartDocumentAnalysisCommand({
DocumentLocation: { S3Object: { Bucket: url.hostname, Name: url.pathname.slice(1) } },
FeatureTypes: features as FeatureType[],
NotificationChannel: {
SNSTopicArn: process.env.TEXTRACT_SNS_TOPIC_ARN!,
RoleArn: process.env.TEXTRACT_NOTIFICATION_ROLE_ARN!,
},
}));
await saveJobMetadata(resp.JobId!, { s3_uri, features });
return { content: [{ type: 'text', text: JSON.stringify({ jobId: resp.JobId }) }] };
});
// Tool 2: poll status (lightweight — MaxResults:1)
server.tool('check_analysis_status', {
job_id: z.string(),
}, async ({ job_id }) => {
const status = await getJobStatus(client, job_id);
return { content: [{ type: 'text', text: JSON.stringify(status) }] };
});
// Tool 3: retrieve results (paginate + extract)
server.tool('get_analysis_results', {
job_id: z.string(),
result_type: z.enum(['text', 'tables', 'forms', 'queries']).default('text'),
}, async ({ job_id, result_type }) => {
const { status } = await getJobStatus(client, job_id);
if (status !== 'SUCCEEDED') {
return { content: [{ type: 'text', text: JSON.stringify({ error: `Not ready: ${status}` }) }] };
}
const blocks = await getAllBlocks(client, job_id);
const result = await extractByType(blocks, result_type); // use patterns 2 or 3 above
return { content: [{ type: 'text', text: JSON.stringify(result) }] };
});
The orchestrator calls start_document_analysis, receives a jobId, then loops on check_analysis_status until SUCCEEDED, then calls get_analysis_results. All three tool handlers return in under one second except the final retrieve, which may take several seconds for large documents. Connect SNS to SQS (fan-out) so the MCP server can poll for completion via SQS rather than requiring an inbound webhook endpoint — the SNS→SQS pattern decouples the document processing pipeline from the real-time MCP tool interface. Default quota: 5 concurrent async jobs per account per region — StartDocumentAnalysis throws LimitExceededException when this is exceeded; implement exponential backoff in the submit tool. See the full async guide for the SNS completion message structure, SQS fan-out Lambda handler, and high-volume job queue architecture.
Production checklist
| Check | Common failure mode if skipped |
|---|---|
| Use async API for any PDF >1 page | Sync API silently drops pages 2+ — no exception thrown, empty results for fields on later pages |
Build id→Block Map before any relationship traversal |
Relationship IDs reference blocks at arbitrary positions — array position access produces wrong blocks |
Pair QueriesConfig with FeatureTypes: [FeatureType.QUERIES] |
Omitting either silently ignores queries and returns no QUERY/QUERY_RESULT blocks |
| Check for missing ANSWER relationship (no QUERY_RESULT) separately from low confidence | Code that calls .find(r => r.Type === 'ANSWER')!.Ids[0] without null check throws on fields not found |
| Fill all spanned positions for merged cells in grid assembly | Grid rows after a merged cell misalign by the span width — produces incorrect column assignments for all subsequent cells |
| Assemble all GetDocumentAnalysis pages before traversal | TABLE→CELL relationships that span response pages produce missing child blocks and empty tables |
Add iam:PassRole on notification role ARN to caller principal |
StartDocumentAnalysis rejected — PassRole is required to pass a role to Textract even if the role itself is correctly configured |
Use aws:SourceAccount condition on Textract notification role trust |
Without it, any AWS account's Textract jobs can assume your notification role (confused-deputy vulnerability) |
| Route Lambda/ECS outbound traffic through NAT or internet gateway | Textract has no VPC endpoint for data-plane APIs — private subnet functions without NAT cannot reach the Textract endpoint |
| Set confidence floor on AnalyzeExpense and Queries results (60 recommended) | Low-confidence extractions reach downstream accounting or identity systems as incorrect values with no signal to flag them |
Cast r.Type as string for TABLE_TITLE/TABLE_FOOTER relationship checks |
Older SDK TypeScript types don't include these as valid enum values — strict enum comparison silently misses title/footer blocks |
| Implement exponential backoff on StartDocumentAnalysis | LimitExceededException at 5 concurrent jobs/region crashes the submit tool handler without retry |