Textract AnalyzeExpense in MCP Servers — invoice parsing, SummaryFields, LineItemGroups

AnalyzeExpense vs AnalyzeDocument FORMS

Both APIs can extract key-value data from financial documents, but they serve different purposes. AnalyzeDocument with FORMS returns generic KEY_VALUE_SET blocks where the "key" is whatever label text appears near the input field — you get back raw string pairs like ("Invoice #", "INV-10042") with no semantic normalization. The label text varies by document template: one supplier uses "Invoice No." while another uses "Invoice Number" and a third uses "Ref. #". You must write matching logic for every variation.

AnalyzeExpense normalizes field types into a controlled vocabulary. It recognizes that "Invoice No.", "Invoice Number", and "Ref. #" all refer to the same concept and labels them all INVOICE_RECEIPT_ID. This semantic normalization makes invoice processing robust to supplier template variation without document-specific configuration. Use AnalyzeExpense whenever the document is a financial document (invoice, receipt, credit note); fall back to AnalyzeDocument FORMS or QUERIES for non-financial forms where AnalyzeExpense's semantic model doesn't apply.

SummaryField type vocabulary

Common SummaryField type strings you'll encounter in production invoices:

A single document may have multiple SummaryFields with the same type — for example, a credit note might have two TAX fields for different tax rates. The array may also be empty for a given type if Textract did not detect that field on the document. Always iterate the entire SummaryFields array; do not assume unique types.

Full implementation: parse an invoice to a structured object

import {
  TextractClient,
  AnalyzeExpenseCommand,
  type ExpenseField,
} from '@aws-sdk/client-textract';

const client = new TextractClient({ region: 'us-east-1' });

interface LineItem {
  description: string;
  quantity: string;
  unitPrice: string;
  amount: string;
  raw: Record<string, string>;
}

interface ParsedInvoice {
  vendorName: string | null;
  vendorAddress: string | null;
  invoiceNumber: string | null;
  invoiceDate: string | null;
  dueDate: string | null;
  paymentTerms: string | null;
  poNumber: string | null;
  subtotal: string | null;
  tax: string | null;
  total: string | null;
  lineItems: LineItem[];
  otherFields: Record<string, string>;
  warnings: string[];
}

const CONFIDENCE_THRESHOLD = 60;

function extractExpenseField(
  field: ExpenseField,
  threshold: number,
): { type: string; value: string } | null {
  const typeText = field.Type?.Text;
  const value = field.ValueDetection?.Text;
  const confidence = field.ValueDetection?.Confidence ?? 0;

  if (!typeText || !value) return null;
  if (confidence < threshold) return null;

  return { type: typeText, value };
}

async function parseInvoice(
  bucket: string,
  key: string,
): Promise<ParsedInvoice> {
  const resp = await client.send(new AnalyzeExpenseCommand({
    Document: { S3Object: { Bucket: bucket, Name: key } },
  }));

  const warnings: string[] = [];
  const expenseDocs = resp.ExpenseDocuments ?? [];

  if (expenseDocs.length === 0) {
    return {
      vendorName: null, vendorAddress: null, invoiceNumber: null,
      invoiceDate: null, dueDate: null, paymentTerms: null, poNumber: null,
      subtotal: null, tax: null, total: null, lineItems: [], otherFields: {},
      warnings: ['AnalyzeExpense returned no expense documents'],
    };
  }

  if (expenseDocs.length > 1) {
    warnings.push(`Multiple expense documents detected (${expenseDocs.length}); processing first only`);
  }

  const expenseDoc = expenseDocs[0];

  // Process summary fields
  const summaryMap: Record<string, string[]> = {};
  const otherFields: Record<string, string> = {};

  for (const field of expenseDoc.SummaryFields ?? []) {
    const extracted = extractExpenseField(field, CONFIDENCE_THRESHOLD);
    if (!extracted) continue;

    if (extracted.type === 'OTHER') {
      // Use LabelDetection text as key for OTHER fields
      const label = field.LabelDetection?.Text ?? `other_${Object.keys(otherFields).length}`;
      otherFields[label] = extracted.value;
    } else {
      if (!summaryMap[extracted.type]) summaryMap[extracted.type] = [];
      summaryMap[extracted.type].push(extracted.value);
    }
  }

  // For fields that can appear multiple times, join with " / "
  const getSummary = (type: string) => summaryMap[type]?.join(' / ') ?? null;

  // Process line items
  const lineItems: LineItem[] = [];
  for (const group of expenseDoc.LineItemGroups ?? []) {
    for (const lineItem of group.LineItems ?? []) {
      const raw: Record<string, string> = {};
      for (const field of lineItem.LineItemExpenseFields ?? []) {
        const extracted = extractExpenseField(field, CONFIDENCE_THRESHOLD);
        if (extracted) raw[extracted.type] = extracted.value;
      }
      lineItems.push({
        description: raw['ITEM'] ?? raw['PRODUCT_CODE'] ?? '',
        quantity: raw['QUANTITY'] ?? '',
        unitPrice: raw['UNIT_PRICE'] ?? '',
        amount: raw['PRICE'] ?? raw['EXPENSE_ROW'] ?? '',
        raw,
      });
    }
  }

  return {
    vendorName: getSummary('VENDOR_NAME'),
    vendorAddress: getSummary('VENDOR_ADDRESS'),
    invoiceNumber: getSummary('INVOICE_RECEIPT_ID'),
    invoiceDate: getSummary('INVOICE_RECEIPT_DATE'),
    dueDate: getSummary('DUE_DATE'),
    paymentTerms: getSummary('PAYMENT_TERMS'),
    poNumber: getSummary('PO_NUMBER'),
    subtotal: getSummary('SUBTOTAL'),
    tax: getSummary('TAX'),
    total: getSummary('TOTAL'),
    lineItems,
    otherFields,
    warnings,
  };
}

LineItemGroups structure and EXPENSE_ROW

LineItemGroups organizes line items into groups. Most invoices have a single group, but some documents contain multiple item sections (e.g., a hardware section and a services section). Each group has a LineItemGroupIndex (1-based) and a LineItems array.

Each LineItem has a LineItemExpenseFields array. Common field type strings for line item fields:

• ITEM — product or service description
• PRODUCT_CODE — SKU or catalog number
• QUANTITY — number of units
• UNIT_PRICE — price per unit
• PRICE — extended price for this line (quantity × unit price)
• EXPENSE_ROW — the raw text of the entire line item row as a single string; useful when individual column extraction fails
• OTHER — unrecognized column; use LabelDetection.Text as the column name

EXPENSE_ROW is a fallback: it contains the full text of the line row concatenated. If quantity, unit price, and amount extraction are all failing (returning empty or null), check whether EXPENSE_ROW contains the row text and parse it with a regex or pass it to a language model for extraction. This is the Textract equivalent of "I can see the row but couldn't parse its columns."

Multi-document PDFs

If a PDF contains multiple financial documents (e.g., a ZIP of invoices converted to a single PDF, or a combined invoice + credit note), AnalyzeExpense may return multiple ExpenseDocuments in a single API call. Each ExpenseDocument has an ExpenseIndex field (1-based) indicating its position in the input. Process each independently:

async function parseAllInvoicesInPdf(
  bucket: string,
  key: string,
): Promise<ParsedInvoice[]> {
  const resp = await client.send(new AnalyzeExpenseCommand({
    Document: { S3Object: { Bucket: bucket, Name: key } },
  }));

  return (resp.ExpenseDocuments ?? []).map((expenseDoc, idx) => {
    const summaryMap: Record<string, string> = {};
    for (const field of expenseDoc.SummaryFields ?? []) {
      const extracted = extractExpenseField(field, CONFIDENCE_THRESHOLD);
      if (extracted && extracted.type !== 'OTHER') {
        summaryMap[extracted.type] = extracted.value;
      }
    }
    // ... process line items same as above ...
    return {
      // minimal example
      vendorName: summaryMap['VENDOR_NAME'] ?? null,
      invoiceNumber: summaryMap['INVOICE_RECEIPT_ID'] ?? null,
      total: summaryMap['TOTAL'] ?? null,
      // additional fields omitted for brevity
    } as unknown as ParsedInvoice;
  });
}

Note that sync AnalyzeExpense supports multi-page PDFs with up to 10 pages per call. For larger PDFs or batch processing scenarios, the async path (via StartExpenseAnalysis / GetExpenseAnalysis) handles up to 500 pages and returns the same ExpenseDocuments structure through paginated GetExpenseAnalysis responses. See the Textract async guide for the SNS notification pattern.

Currency amount parsing

Textract returns currency amounts as raw strings exactly as they appear on the document: "$1,512.00", "USD 1,512.00", "1.512,00" (European comma-as-decimal-separator format), or "1512" (plain integer). Do not assume a fixed format. For downstream accounting system integration, parse these strings to numeric values in your MCP tool rather than passing raw strings:

function parseAmount(raw: string | null): number | null {
  if (!raw) return null;

  // Remove currency symbols and whitespace
  let s = raw.replace(/[^\d.,\-]/g, '').trim();

  // Detect European format: 1.234,56 (period as thousands sep, comma as decimal)
  if (/^\d{1,3}(\.\d{3})*(,\d+)?$/.test(s)) {
    s = s.replace(/\./g, '').replace(',', '.');
  } else {
    // US/UK format: 1,234.56 (comma as thousands sep, period as decimal)
    s = s.replace(/,/g, '');
  }

  const num = parseFloat(s);
  return isNaN(num) ? null : num;
}

// Usage
const totalAmount = parseAmount(invoice.total);  // null if not extractable

This is defensive — it returns null rather than NaN for values that cannot be parsed. MCP tools should surface null amounts to the orchestrator rather than silently passing zero or NaN to downstream accounting systems.

MCP tool pattern for AP automation

A complete accounts-payable MCP tool set has three tools:

1. analyze_invoice — takes an S3 URI, calls AnalyzeExpense, returns structured JSON with summary fields and line items. This is the extraction step.

2. validate_invoice — takes the structured invoice object, runs validation rules (required fields present, line item amounts sum to subtotal, subtotal + tax = total, vendor exists in vendor master), returns a validation report. This is the rules step.

3. post_invoice — takes a validated invoice object and a GL account mapping, posts to the accounting API (QuickBooks, NetSuite, etc.). This is the action step.

The AI orchestrator calls these in sequence: extract → validate → human approval on exceptions → post. The MCP server handles the document intelligence; the orchestrator handles routing and exception escalation. Keeping Textract calls isolated in the extraction tool makes it easy to swap in a different extraction provider (e.g., Azure Document Intelligence) without changing the downstream tools.