AWS Comprehend · 2026-11-03 · Comprehend Advanced arc

AWS Comprehend Advanced NLP for MCP Servers: PII Detection, Custom Entity Recognition, Custom Classification, and Events Extraction

AliveMCP monitors MCP endpoints every 60 seconds — and the endpoints that produce the most ambiguous failure signals are NLP processing tools, because language model APIs fail softly: Comprehend's ContainsPiiEntities returns an empty Labels array when the input exceeds 5,000 bytes rather than throwing, a custom entity recognizer deployed against unrelated text returns an empty Entities array without indicating that the model was invoked on out-of-distribution input, a multi-class classifier at 0.52 confidence routes to the wrong handler with no visible error, and an Events detection job with a misspelled event type silently runs without detecting any events. Four Comprehend capabilities close the gap between a prototype NLP tool and a production-grade text intelligence MCP server. PII detection addresses the most urgent safety concern: MCP servers that process user-supplied text must guard against forwarding personally identifiable information — and credentials — to downstream systems and logs before consent gates are in place; ContainsPiiEntities provides a fast binary pre-screen (returning Labels[] with entity type and score, no spans, responding in under 200ms) while DetectPiiEntities returns exact character offsets for redaction, and the async StartPiiEntitiesDetectionJob processes S3 document batches in either ONLY_REDACTION mode (producing a copy of the document with PII masked character-for-character) or ONLY_OFFSETS mode (returning coordinates without modifying the document); critically, Comprehend detects AWS_ACCESS_KEY and AWS_SECRET_KEY as PII entity types — users occasionally paste AWS credentials into tool arguments, and a PII gate before logging catches this before it reaches CloudWatch logs. Custom entity recognition addresses the limits of built-in NER: Comprehend's standard DetectEntities identifies 12 general named entity types — PERSON, LOCATION, ORGANIZATION, DATE, and others — but cannot identify "alivemcp-status-server" as an MCP_SERVER_SLUG or "CVE-2026-12345" as a VULNERABILITY_ID; custom recognizers train on your annotated documents and learn domain-specific entity patterns, with two annotation formats (CSV with tab-separated offset spans for manually-curated small datasets, and augmented manifests from SageMaker Ground Truth for large annotation workflows), real-time endpoint deployment at 100 characters per second per inference unit, and async batch via StartEntitiesDetectionJobCommand without endpoint overhead. Custom classification provides the routing layer: instead of branching on keywords or calling a general-purpose LLM for routing decisions, a trained Comprehend classifier applies a purpose-built model — either MULTI_CLASS (softmax across labels; output scores sum to 1.0; pick Classes[0]) for mutually exclusive intent routing, or MULTI_LABEL (independent probability per label; scores do not sum to 1.0; apply per-label threshold) for topic tagging — and exposes it via the same real-time endpoint or async batch path as entity recognizers. Events detection takes NLP output from entities to structured knowledge: instead of returning "AWS acquired Anthropic" as two ORGANIZATION entities and a verb, Events detection returns a structured ACQUISITION event with SUBJECT=AWS, OBJECT=Anthropic, DATE_TIME, and ARTIFACT roles linked by EntityId references into the accompanying Entities array — output that an AI orchestrator can reason over directly without re-parsing. This guide synthesizes all four Comprehend patterns into a complete NLP MCP server stack. Pattern 1 covers PII detection and the credential-leak gate. Pattern 2 covers custom entity recognizer training, deployment, and inference. Pattern 3 covers document classification in both modes with real-time and batch paths. Pattern 4 covers Events detection with role semantics and the two-tool async MCP pattern.

TL;DR

Pattern 1: PII Detection — the credential-leak gate every MCP server needs

MCP servers that accept user-supplied text — document uploads, natural-language queries, support ticket text, chat messages — face a compliance and security risk that manifests before any business logic runs: the user may have included personally identifiable information or credentials in their input. Forwarding this input directly to external APIs, writing it to CloudWatch Logs, or passing it to a downstream AI model before consent checks can expose PII to systems that were never authorized to process it. Comprehend PII detection provides two complementary APIs for different points in this pipeline.

ContainsPiiEntities: the fast pre-screen gate

ContainsPiiEntitiesCommand answers a binary question — does this document contain PII? — without returning character offsets. It typically responds in under 200ms and is the right tool for pre-screening user input before forwarding to external services:

import {
  ComprehendClient,
  ContainsPiiEntitiesCommand,
  DetectPiiEntitiesCommand,
  LanguageCode,
  type PiiEntity,
} from '@aws-sdk/client-comprehend';

const client = new ComprehendClient({ region: 'us-east-1' });

// Types that hard-block processing — never forward these downstream
const HARD_BLOCK_TYPES = new Set([
  'SSN', 'CREDIT_DEBIT_NUMBER', 'BANK_ACCOUNT_NUMBER',
  'AWS_ACCESS_KEY', 'AWS_SECRET_KEY', 'PASSWORD', 'PASSPORT_NUMBER',
  'DRIVER_ID',
]);

interface PiiScreenResult {
  containsPii: boolean;
  hardBlock: boolean;
  detectedTypes: Array<{ type: string; score: number }>;
}

async function screenForPii(text: string, threshold = 0.80): Promise<PiiScreenResult> {
  // Split long text — 5,000 UTF-8 byte hard limit per API call
  const chunks = splitIntoChunks(text, 4500);
  const allDetected: Array<{ type: string; score: number }> = [];

  for (const chunk of chunks) {
    const { Labels } = await client.send(new ContainsPiiEntitiesCommand({
      Text: chunk,
      LanguageCode: LanguageCode.EN,
    }));
    for (const label of Labels ?? []) {
      if ((label.Score ?? 0) >= threshold) {
        allDetected.push({ type: label.Name ?? '', score: label.Score ?? 0 });
      }
    }
  }

  return {
    containsPii: allDetected.length > 0,
    hardBlock: allDetected.some(d => HARD_BLOCK_TYPES.has(d.type)),
    detectedTypes: allDetected,
  };
}

function splitIntoChunks(text: string, maxBytes: number): string[] {
  const encoder = new TextEncoder();
  if (encoder.encode(text).length <= maxBytes) return [text];

  const chunks: string[] = [];
  const paragraphs = text.split(/\n\n+/);
  let current = '';

  for (const para of paragraphs) {
    const candidate = current ? `${current}\n\n${para}` : para;
    if (encoder.encode(candidate).length > maxBytes) {
      if (current) chunks.push(current);
      current = para;
    } else {
      current = candidate;
    }
  }
  if (current) chunks.push(current);
  return chunks;
}

The key distinction: ContainsPiiEntities returns Labels[] with entity type names and confidence scores only — it does NOT return character offsets. It is fast because it performs classification rather than extraction. Use it at the tool handler entry point before any other processing.

DetectPiiEntities: span-level extraction and right-to-left redaction

When you need to know exactly where the PII appears — to redact it, log only the non-PII portions, or present an audit trail — call DetectPiiEntitiesCommand. The critical implementation detail: always process spans right-to-left when building redacted text:

interface RedactionResult {
  redactedText: string;
  detectedEntities: Array<{ type: string; score: number; original: string }>;
}

async function detectAndRedact(text: string, threshold = 0.80): Promise<RedactionResult> {
  const { Entities } = await client.send(new DetectPiiEntitiesCommand({
    Text: text,
    LanguageCode: LanguageCode.EN,
  }));

  // Filter by confidence and validate offset fields exist
  const confident = (Entities ?? [])
    .filter((e): e is Required<PiiEntity> =>
      (e.Score ?? 0) >= threshold &&
      e.BeginOffset !== undefined &&
      e.EndOffset !== undefined
    )
    .sort((a, b) => a.BeginOffset - b.BeginOffset);

  let redacted = text;
  const detected: RedactionResult['detectedEntities'] = [];

  // Process in reverse offset order to prevent offset drift
  // Replacing a span at offset 200-210 changes string length —
  // all offsets below 200 remain valid, but forward replacements would
  // reference wrong positions after the first substitution
  for (const entity of [...confident].reverse()) {
    const original = text.slice(entity.BeginOffset, entity.EndOffset);
    const mask = `[${entity.Type}]`;
    redacted =
      redacted.slice(0, entity.BeginOffset) +
      mask +
      redacted.slice(entity.EndOffset);
    detected.unshift({ type: entity.Type, score: entity.Score, original });
  }

  return { redactedText: redacted, detectedEntities: detected };
}

The right-to-left requirement is non-obvious and easy to miss: if you process spans left-to-right and the mask string [NAME] is shorter than the original span "John Bartholomew Smith", the byte positions of all subsequent spans shift. Sorting descending by BeginOffset and replacing from the end of the string forward eliminates this entirely — lower-offset spans are unaffected by replacements at higher offsets.

The AWS credential detection use case

The most operationally significant PII types for MCP server operators are AWS_ACCESS_KEY and AWS_SECRET_KEY. Users paste credentials into tool arguments more often than you'd expect — copying a command from a tutorial, testing with temporary credentials, or using an MCP tool to process a configuration file that contains credentials. Comprehend detects AKIA...-prefixed access key IDs and 40-character secret keys reliably:

import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
import { z } from 'zod';

const server = new McpServer({ name: 'document-processor', version: '1.0.0' });

server.tool('process_document', {
  document_text: z.string().max(4000),
}, async ({ document_text }) => {
  const screen = await screenForPii(document_text);

  if (screen.hardBlock) {
    // Hard block — credentials or financial PII detected
    // Do NOT log document_text here — that would defeat the gate
    const types = screen.detectedTypes
      .filter(t => HARD_BLOCK_TYPES.has(t.type))
      .map(t => t.type);

    return {
      content: [{
        type: 'text',
        text: JSON.stringify({
          error: 'Document contains sensitive PII that cannot be processed',
          detected_types: types,
          action: types.some(t => t.includes('KEY') || t === 'PASSWORD')
            ? 'Credentials detected — rotate immediately if these are real keys'
            : 'Remove PII before resubmitting',
        }),
      }],
    };
  }

  if (screen.containsPii) {
    // Soft PII detected — redact before processing
    const { redactedText } = await detectAndRedact(document_text);
    return processRedacted(redactedText);
  }

  return processRedacted(document_text);
});

Async PII redaction for batch document pipelines

For batch workloads — nightly processing of accumulated document uploads, compliance-required redaction of an existing document archive — the async StartPiiEntitiesDetectionJobCommand processes S3 document sets without the 5,000-byte per-call limit:

import {
  StartPiiEntitiesDetectionJobCommand,
  DescribePiiEntitiesDetectionJobCommand,
} from '@aws-sdk/client-comprehend';

const { JobId } = await client.send(new StartPiiEntitiesDetectionJobCommand({
  JobName: `pii-redact-${Date.now()}`,
  DataAccessRoleArn: 'arn:aws:iam::123456789012:role/ComprehendDataAccessRole',
  LanguageCode: LanguageCode.EN,
  Mode: 'ONLY_REDACTION',
  RedactionConfig: {
    // REPLACE_WITH_PII_ENTITY_TYPE produces "[NAME]", "[EMAIL]" markers
    // REDACT produces asterisk-filled spans: "***" — less human-readable
    MaskMode: 'REPLACE_WITH_PII_ENTITY_TYPE',
    // Optional: only redact specific types
    PiiEntityTypes: ['NAME', 'EMAIL', 'SSN', 'AWS_ACCESS_KEY', 'AWS_SECRET_KEY'],
  },
  InputDataConfig: {
    S3Uri: 's3://my-bucket/raw-documents/',
    InputFormat: 'ONE_DOC_PER_LINE',
  },
  OutputDataConfig: {
    S3Uri: 's3://my-bucket/redacted-documents/',
  },
}));

The two mask modes produce different outputs: REPLACE_WITH_PII_ENTITY_TYPE replaces each span with its entity type label ([NAME], [AWS_ACCESS_KEY]) — preserving document structure and making the redaction auditable; REDACT replaces each span with asterisks (***) matching the original span length — appropriate when the entity type classification itself must not be disclosed. For compliance audit trails, REPLACE_WITH_PII_ENTITY_TYPE is almost always the right choice because it lets reviewers verify that redaction was complete without revealing the original values.

Full list of Comprehend PII entity types, pricing, and the MCP pre-screening gate pattern are covered in detail in AWS Comprehend PII Detection in MCP Servers.

Pattern 2: Custom Entity Recognition — training domain-specific NER models for MCP tool extraction

Comprehend's built-in entity recognizer is a general-purpose NER model. It correctly identifies standard named entities — people, places, organizations, dates, quantities — but it has never seen your domain vocabulary. An MCP server that needs to extract MCP server slugs from freeform issue text, parse CVE identifiers from security advisories, or identify proprietary product codes from support tickets cannot use DetectEntities for those extraction tasks. Custom entity recognizers fill this gap.

Designing your entity type list

Before writing annotation data, design the entity type list carefully. Comprehend supports up to 25 custom entity types per recognizer. Entity type names must match the regex [A-Z_]+ — uppercase letters and underscores only, no digits, hyphens, or spaces.

Design principleGood exampleBad example
Types map to distinct downstream handlingMCP_SERVER_SLUG, REGISTRY_NAME, TOOL_ERROR_CODEPRODUCT (catches everything)
Types are mutually exclusive in practiceCVE_ID, ADVISORY_IDIDENTIFIER (too broad)
Each type has consistent surface patternsSERVER_VERSION (e.g., "1.2.3")STATUS_INFO (highly variable)
You can provide 200+ examples per typeAny type you actually labelA type you can only annotate 20 times

Too few types and precision suffers — a single PRODUCT type for all product-related text learns vague patterns. Too many types and you need far more annotation data per type to train a good model. A workable rule: if you would write separate code to handle two entity types differently downstream, they warrant separate types in the recognizer.

CSV annotation format

The CSV format requires two S3 objects: a documents file (one document per line, plain UTF-8 text) and a tab-separated annotations file:

# documents.txt — one document per line in S3
The mcp-server-filesystem endpoint has been down for 3 hours per the AliveMCP dashboard.
User reports mcp-server-brave-search returned TOOL_ERROR on all calls since deployment.
AliveMCP shows mcp-server-sequential-thinking at 99.9% uptime over the last 30 days.
CVE-2026-12345 affects all mcp-server-memory versions below 2.1.0.

# annotations.csv — tab-separated (use actual tabs, not spaces)
File	Line	Begin Offset	End Offset	Type
s3://my-bucket/train/documents.txt	0	4	26	MCP_SERVER_SLUG
s3://my-bucket/train/documents.txt	1	18	41	MCP_SERVER_SLUG
s3://my-bucket/train/documents.txt	1	50	60	TOOL_ERROR_CODE
s3://my-bucket/train/documents.txt	2	9	38	MCP_SERVER_SLUG
s3://my-bucket/train/documents.txt	3	0	14	CVE_ID
s3://my-bucket/train/documents.txt	3	42	63	MCP_SERVER_SLUG

Offset precision is the most common source of poor model quality. Off-by-one errors — annotating "mcp-server-filesystem " (trailing space) instead of "mcp-server-filesystem" — are systematic errors that consistently degrade F1 scores. When building annotation pipelines, always verify annotation spans with a snippet extraction test: line.slice(beginOffset, endOffset) should exactly equal the entity text.

Creating and training the recognizer

import {
  ComprehendClient,
  CreateEntityRecognizerCommand,
  DescribeEntityRecognizerCommand,
  LanguageCode,
} from '@aws-sdk/client-comprehend';

const client = new ComprehendClient({ region: 'us-east-1' });

const { EntityRecognizerArn } = await client.send(
  new CreateEntityRecognizerCommand({
    RecognizerName: 'mcp-server-entities-v1',
    LanguageCode: LanguageCode.EN,
    DataAccessRoleArn: 'arn:aws:iam::123456789012:role/ComprehendDataAccessRole',
    InputDataConfig: {
      DataFormat: 'COMPREHEND_CSV',
      EntityTypes: [
        { Type: 'MCP_SERVER_SLUG' },
        { Type: 'REGISTRY_NAME' },
        { Type: 'TOOL_ERROR_CODE' },
        { Type: 'CVE_ID' },
      ],
      Documents: { S3Uri: 's3://my-bucket/train/documents.txt' },
      Annotations: { S3Uri: 's3://my-bucket/train/annotations.csv' },
    },
  })
);

// Training is async — poll until TRAINED (30–90 minutes)
async function waitForTraining(arn: string) {
  while (true) {
    const { EntityRecognizerProperties: props } = await client.send(
      new DescribeEntityRecognizerCommand({ EntityRecognizerArn: arn })
    );

    if (props?.Status === 'TRAINED') {
      const metrics = props.RecognizerMetadata?.EvaluationMetrics;
      // Production gate: F1 >= 0.80
      // If below threshold, check per-type metrics in RecognizerMetadata.EntityTypes[]
      // Low-F1 types need more annotation examples or better annotation consistency
      console.log('F1:', metrics?.F1Score, 'Precision:', metrics?.Precision, 'Recall:', metrics?.Recall);
      return arn;
    }
    if (props?.Status === 'IN_ERROR') {
      throw new Error(`Training failed: ${props.Message ?? 'unknown'}`);
    }
    await new Promise(r => setTimeout(r, 60_000));
  }
}

The DataAccessRoleArn requires a trust policy allowing comprehend.amazonaws.com to assume it — always include an aws:SourceAccount condition on the trust policy to prevent confused deputy attacks where Comprehend is tricked into assuming your role for another customer's job. The role needs s3:GetObject and s3:ListBucket on the training bucket.

Endpoint deployment and real-time inference in MCP tool handlers

import {
  CreateEndpointCommand,
  DescribeEndpointCommand,
  DetectEntitiesCommand,
} from '@aws-sdk/client-comprehend';
import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
import { z } from 'zod';

// Deploy endpoint — 1 inference unit = 100 chars/sec sustained throughput
const { EndpointArn } = await client.send(new CreateEndpointCommand({
  EndpointName: 'mcp-entities-prod',
  ModelArn: entityRecognizerArn,
  DesiredInferenceUnits: 1,
}));

// Wait for IN_SERVICE (5–15 minutes)
await waitForEndpoint(EndpointArn);

const server = new McpServer({ name: 'entity-extractor', version: '1.0.0' });
const ENTITY_CONFIDENCE_THRESHOLD = 0.80;

server.tool('extract_entities', {
  text: z.string().max(4000),
  entity_types: z.array(z.string()).optional(),
}, async ({ text, entity_types }) => {
  const { Entities } = await client.send(new DetectEntitiesCommand({
    Text: text,
    EndpointArn: process.env.COMPREHEND_ENTITY_ENDPOINT_ARN!,
  }));

  const extracted = (Entities ?? [])
    .filter(e =>
      (e.Score ?? 0) >= ENTITY_CONFIDENCE_THRESHOLD &&
      (!entity_types || entity_types.includes(e.Type ?? ''))
    )
    .map(e => ({
      text: e.Text,
      type: e.Type,
      score: e.Score,
      begin: e.BeginOffset,
      end: e.EndOffset,
    }));

  // Deduplicate overlapping spans — keep highest-confidence when spans overlap
  const deduped = deduplicateOverlappingSpans(extracted);

  return {
    content: [{ type: 'text', text: JSON.stringify({ entities: deduped }) }],
  };
});

Async batch recognition without an endpoint

For batch document processing tasks where real-time latency is not required, use StartEntitiesDetectionJobCommand with the recognizer ARN directly — no endpoint deployment needed, and endpoint hourly charges don't apply:

import { StartEntitiesDetectionJobCommand } from '@aws-sdk/client-comprehend';

const { JobId } = await client.send(new StartEntitiesDetectionJobCommand({
  JobName: `entity-batch-${Date.now()}`,
  EntityRecognizerArn: entityRecognizerArn,
  DataAccessRoleArn: 'arn:aws:iam::123456789012:role/ComprehendDataAccessRole',
  LanguageCode: LanguageCode.EN,
  InputDataConfig: {
    S3Uri: 's3://my-bucket/documents/',
    InputFormat: 'ONE_DOC_PER_LINE',
  },
  OutputDataConfig: { S3Uri: 's3://my-bucket/entity-output/' },
}));

// Output: one JSON line per document
// {"Entities": [{"BeginOffset": 4, "EndOffset": 26, "Score": 0.93, "Text": "mcp-server-filesystem", "Type": "MCP_SERVER_SLUG"}]}

Cost comparison: endpoint inference is billed per character processed ($0.0005/100 chars) plus endpoint hours ($0.0005/hour per IU). A 24/7 endpoint at 1 IU costs ~$0.36/day regardless of volume. Async batch is billed per character only — for irregular or low-volume workloads, async batch is dramatically cheaper. Train patterns, confidence tuning, augmented manifest format, and per-type F1 diagnosis are covered in depth in AWS Comprehend Custom Entity Recognition in MCP Servers.

Pattern 3: Custom Classification — intent routing and document tagging with trained models

Routing MCP tool calls — sending server-unreachable issues to one handler, auth failures to another, schema drift to a third — is easy to prototype with keyword matching and fragile in production. Keyword matchers miss paraphrases, misfire on unrelated mentions, and require manual maintenance as vocabulary evolves. Custom Comprehend classifiers provide a trained model that generalizes across paraphrase variation and produces calibrated confidence scores that you can use to detect out-of-distribution inputs.

MULTI_CLASS vs MULTI_LABEL: the architectural choice

DimensionMULTI_CLASSMULTI_LABEL
Labels per documentExactly one winnerZero or more
Output scoresSum to 1.0 (softmax)Independent 0–1 per label
Routing decisionPick Classes[0]Threshold each label independently
Training minimum10 examples/class5 examples/label
Training CSV formatCLASS,TEXT"L1|L2",TEXT
Best forIntent routing, ticket triage, language detectionTopic tagging, content labeling, multi-topic documents
Low-confidence signalClasses[0].Score < 0.70 → fallback handlerNo label scores above threshold → untagged

For AliveMCP's primary use case — routing MCP server issue reports to the right handler — MULTI_CLASS is correct: each issue should route to exactly one primary category (SERVER_DOWN, AUTH_FAILURE, SCHEMA_DRIFT, CERT_EXPIRED, RATE_LIMITED, TIMEOUT). MULTI_LABEL would be appropriate for tagging blog posts or documentation with multiple applicable topics.

Training data: both CSV formats

# MULTI_CLASS format: CLASS,TEXT (no header row)
# Text with commas must be double-quoted
SERVER_DOWN,"The mcp-server-filesystem endpoint is not responding. AliveMCP shows 45 minutes down."
SERVER_DOWN,Getting connection refused on all calls to mcp-server-brave-search since 14:30 UTC.
AUTH_FAILURE,mcp-server-github returns 401 on every call. Credentials look correct from my side.
AUTH_FAILURE,"Authentication failing for mcp-server-gitlab. Token rotated yesterday but error persists."
SCHEMA_DRIFT,"mcp-server-postgres tool schema changed, orchestrator can't find query_table anymore."
CERT_EXPIRED,"SSL certificate error on mcp-server-everything. Browser also shows cert warning."
RATE_LIMITED,"Getting 429 Too Many Requests from mcp-server-openai since the traffic spike."
TIMEOUT,"mcp-server-memory calls hanging for 30+ seconds then timing out. Started 2 hours ago."

# MULTI_LABEL format: "L1|L2",TEXT (labels pipe-separated, quoted when multiple)
monitoring,"How do I set up uptime monitoring for my MCP server?"
"monitoring|alerting","Configure webhook alerts when an MCP server goes down."
"monitoring|metrics","What metrics does AliveMCP track for each MCP endpoint?"
"alerting|slack","How do I connect AliveMCP Slack alerts to my workspace?"

Training and deploying the classifier

import {
  ComprehendClient,
  CreateDocumentClassifierCommand,
  ClassifyDocumentCommand,
  DocumentClassifierMode,
  LanguageCode,
} from '@aws-sdk/client-comprehend';

const { DocumentClassifierArn } = await client.send(
  new CreateDocumentClassifierCommand({
    DocumentClassifierName: 'mcp-issue-classifier',
    LanguageCode: LanguageCode.EN,
    Mode: DocumentClassifierMode.MULTI_CLASS,
    VersionName: 'v1-2026-11-03',  // becomes part of the ARN
    DataAccessRoleArn: 'arn:aws:iam::123456789012:role/ComprehendDataAccessRole',
    InputDataConfig: {
      DataFormat: 'COMPREHEND_CSV',
      S3Uri: 's3://my-bucket/training/classifier-train.csv',
    },
    OutputDataConfig: {
      // Confusion matrix and per-class F1 written here after training
      S3Uri: 's3://my-bucket/classifier-output/',
    },
  })
);

// Same endpoint lifecycle as entity recognizers
// CreateEndpointCommand → wait IN_SERVICE → ClassifyDocumentCommand

Real-time classification in MCP tool handlers

import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
import { z } from 'zod';

const server = new McpServer({ name: 'issue-router', version: '1.0.0' });

const CLASSIFICATION_THRESHOLD = 0.70;
const FALLBACK_CLASS = 'UNKNOWN';

server.tool('classify_issue', {
  issue_text: z.string().max(4000),
}, async ({ issue_text }) => {
  const { Classes, Labels } = await client.send(new ClassifyDocumentCommand({
    Text: issue_text,
    EndpointArn: process.env.COMPREHEND_CLASSIFIER_ARN!,
  }));

  // MULTI_CLASS: Classes[] sorted by Score descending
  // Classes[0] is always the top prediction — do not sort manually
  const topClass = Classes?.[0];

  if (!topClass || (topClass.Score ?? 0) < CLASSIFICATION_THRESHOLD) {
    return {
      content: [{
        type: 'text',
        text: JSON.stringify({
          classification: FALLBACK_CLASS,
          confidence: topClass?.Score ?? 0,
          all_classes: (Classes ?? []).slice(0, 3).map(c => ({
            name: c.Name,
            score: c.Score,
          })),
          message: 'Issue does not match known categories with sufficient confidence',
        }),
      }],
    };
  }

  return {
    content: [{
      type: 'text',
      text: JSON.stringify({
        classification: topClass.Name,
        confidence: topClass.Score,
        top_classes: (Classes ?? []).slice(0, 3).map(c => ({
          name: c.Name,
          score: c.Score,
        })),
      }),
    }],
  };
});

The low-confidence fallback route is critical for production reliability. A classifier forced to produce a routing decision at 0.52 confidence routes to the wrong handler roughly half the time — the same as random assignment for a binary problem. Return UNKNOWN and escalate to a human or LLM fallback handler when Classes[0].Score < 0.70.

Async batch classification and cost comparison

import { StartDocumentClassificationJobCommand } from '@aws-sdk/client-comprehend';

// No endpoint required — same per-character cost, no endpoint surcharge
const { JobId } = await client.send(new StartDocumentClassificationJobCommand({
  JobName: `classify-batch-${Date.now()}`,
  DocumentClassifierArn: documentClassifierArn,
  DataAccessRoleArn: 'arn:aws:iam::123456789012:role/ComprehendDataAccessRole',
  InputDataConfig: {
    S3Uri: 's3://my-bucket/batch-input/',
    InputFormat: 'ONE_DOC_PER_LINE',
  },
  OutputDataConfig: { S3Uri: 's3://my-bucket/batch-output/' },
}));
// Output JSONL: {"Line": 0, "Classes": [{"Name": "SERVER_DOWN", "Score": 0.94}, ...]}
PathPer-character costEndpoint costBest for
Real-time endpoint$0.0005/100 chars~$0.0005/hour per IUInteractive MCP tools, <200ms latency requirement
Async batch job$0.0005/100 charsNoneNightly bulk triage, large document backlogs

In-place model version upgrades

UpdateEndpointCommand accepts a new DesiredModelArn pointing at a retrained classifier version. The update takes 5–10 minutes and has no traffic interruption or cold-start window — the endpoint continues serving traffic from the old model until the transition completes. This is the correct zero-downtime upgrade path:

import { UpdateEndpointCommand } from '@aws-sdk/client-comprehend';

// Train new version (same DocumentClassifierName, new VersionName)
// Then upgrade the live endpoint to the new version in-place:
await client.send(new UpdateEndpointCommand({
  EndpointArn: productionEndpointArn,
  DesiredModelArn: newClassifierVersionArn,
  // DesiredInferenceUnits: 2,  // optionally resize at the same time
}));

Multi-version classifiers (same name, different VersionName values) support A/B testing: deploy the new version to a canary endpoint at a fraction of traffic, compare routing accuracy and Classes[0].Score distributions, then upgrade the production endpoint and delete the canary. AWS Comprehend Custom Classification in MCP Servers covers the full classifier lifecycle including confusion matrix interpretation and class imbalance handling.

Pattern 4: Events Detection — structured knowledge extraction from financial and operational text

Comprehend Events detection bridges the gap between named entity recognition and structured knowledge. Where DetectEntities returns "AWS" and "Anthropic" as organizations and "April 1, 2026" as a date, Events detection returns a structured ACQUISITION event: SUBJECT=AWS (the acquirer), OBJECT=Anthropic (the acquired), DATE_TIME=April 1, 2026. The difference in downstream usefulness is substantial — an AI orchestrator can reason over the structured event representation directly without re-parsing natural language.

Supported event types and their role semantics

Events detection is built-in — no custom model training required. It supports 11 event types across financial, employment, and corporate domains, and is available for English only:

Event TypePrimary rolesTypical trigger words
ACQUISITIONSUBJECT (acquirer), OBJECT (acquired), DATE_TIME, ARTIFACT (deal value)acquired, bought, purchased
CORPORATE_ACQUISITIONSUBJECT (acquirer), OBJECT (target), DATE_TIME, ARTIFACTacquired by, taken over, merger
EMPLOYMENTSUBJECT (person), OBJECT (org), ARTIFACT (title), DATE_TIMEjoined, hired, appointed, named
BOARD_CHANGESUBJECT (person), OBJECT (company), ARTIFACT (role)elected, resigned, stepped down, appointed
IPOSUBJECT (company), DATE_TIME, ARTIFACT (exchange, price)priced, went public, listed, IPO
BANKRUPTCYSUBJECT (filing entity), DATE_TIME, ARTIFACT (chapter type)filed for, Chapter 11, Chapter 7
BUSINESS_EXPANSIONSUBJECT (company), ARTIFACT (location/product), DATE_TIMEexpanded, launched, opened, extended
BUSINESS_REDUCTIONSUBJECT (company), ARTIFACT (scope), DATE_TIMElaid off, cut, reduced, downsized
DISSOLVEDSUBJECT (entity), DATE_TIMEdissolved, shut down, closed, wound up
RIGHTS_ISSUESUBJECT (company), ARTIFACT (ratio, price), DATE_TIMErights issue, rights offering
SECONDARY_OFFERINGSUBJECT (company), ARTIFACT (shares, price), DATE_TIMEsecondary offering, follow-on, dilutive

ACQUISITION and CORPORATE_ACQUISITION both detect acquisition events but from different textual perspectives — ACQUISITION typically fires when the acquiring entity is the grammatical subject ("AWS acquired Anthropic"), while CORPORATE_ACQUISITION fires when the target or the transaction itself is foregrounded ("Anthropic was acquired by AWS"). In practice, both may fire for the same underlying event depending on how the text is written. Your extraction logic should handle both types and deduplicate events with overlapping trigger text spans.

Submitting an events detection job

Events detection is async-only — there is no synchronous DetectEvents API. Always structure MCP tools around the two-step submit/retrieve pattern:

import {
  ComprehendClient,
  StartEventsDetectionJobCommand,
  DescribeEventsDetectionJobCommand,
  LanguageCode,
} from '@aws-sdk/client-comprehend';

const client = new ComprehendClient({ region: 'us-east-1' });

const { JobId } = await client.send(new StartEventsDetectionJobCommand({
  JobName: `events-detect-${Date.now()}`,
  DataAccessRoleArn: 'arn:aws:iam::123456789012:role/ComprehendDataAccessRole',
  LanguageCode: LanguageCode.EN,
  // Specify only relevant types — reduces output noise
  // Cost is per-character regardless of event type count
  TargetEventTypes: [
    'ACQUISITION',
    'CORPORATE_ACQUISITION',
    'EMPLOYMENT',
    'BOARD_CHANGE',
    'BANKRUPTCY',
    'BUSINESS_EXPANSION',
    'BUSINESS_REDUCTION',
  ],
  InputDataConfig: {
    S3Uri: 's3://my-bucket/news-articles/',
    InputFormat: 'ONE_DOC_PER_LINE',
  },
  OutputDataConfig: {
    S3Uri: 's3://my-bucket/events-output/',
  },
}));

Output JSON structure and EntityId cross-references

The output JSON Lines file has one object per input document. Each document object contains two top-level arrays:

// Example output for: "AWS acquired Anthropic for $4B in a deal closed on April 1, 2026"
{
  "Entities": [
    { "Id": 0, "Text": "AWS",      "Type": "ORGANIZATION", "BeginOffset": 0,  "EndOffset": 3  },
    { "Id": 1, "Text": "Anthropic","Type": "ORGANIZATION", "BeginOffset": 13, "EndOffset": 22 },
    { "Id": 2, "Text": "$4B",      "Type": "QUANTITY",     "BeginOffset": 27, "EndOffset": 30 },
    { "Id": 3, "Text": "April 1, 2026", "Type": "DATE",   "BeginOffset": 52, "EndOffset": 65 }
  ],
  "Events": [
    {
      "Type": "ACQUISITION",
      "Triggers": [
        { "Id": 4, "Text": "acquired", "Type": "ACQUISITION", "BeginOffset": 4, "EndOffset": 12, "GroupScore": 0.97 }
      ],
      "Arguments": [
        { "EntityId": 0, "Role": "SUBJECT",   "Score": 0.98 },
        { "EntityId": 1, "Role": "OBJECT",    "Score": 0.96 },
        { "EntityId": 2, "Role": "ARTIFACT",  "Score": 0.87 },
        { "EntityId": 3, "Role": "DATE_TIME", "Score": 0.95 }
      ]
    }
  ]
}

Key structural points for parsing:

TypeScript parsing with independent confidence thresholds

interface ComprehendEntity {
  Id: number;
  Text: string;
  Type: string;
  BeginOffset: number;
  EndOffset: number;
}

interface ComprehendEvent {
  Type: string;
  Triggers: Array<{ Id: number; Text: string; GroupScore: number }>;
  Arguments: Array<{ EntityId: number; Role: string; Score: number }>;
}

interface DocumentEventsOutput {
  Entities: ComprehendEntity[];
  Events: ComprehendEvent[];
}

interface StructuredEvent {
  type: string;
  trigger: string;
  triggerConfidence: number;
  subject?: string;
  object?: string;
  date?: string;
  artifact?: string;
  investor?: string;
  recipient?: string;
  raw: ComprehendEvent;
}

// Two independent thresholds:
// - TRIGGER_THRESHOLD: confidence that the event actually happened
// - ROLE_THRESHOLD: confidence that a specific entity fills a role
const TRIGGER_THRESHOLD = 0.70;
const ROLE_THRESHOLD = 0.75;

function parseDocumentEvents(output: DocumentEventsOutput): StructuredEvent[] {
  // Build integer-keyed entity map for fast role resolution
  const entityMap = new Map<number, ComprehendEntity>(
    output.Entities.map(e => [e.Id, e])
  );

  const results: StructuredEvent[] = [];

  for (const event of output.Events) {
    // Use the highest-confidence trigger as the event anchor
    const primaryTrigger = [...event.Triggers].sort(
      (a, b) => b.GroupScore - a.GroupScore
    )[0];

    if (!primaryTrigger || primaryTrigger.GroupScore < TRIGGER_THRESHOLD) continue;

    const structured: StructuredEvent = {
      type: event.Type,
      trigger: primaryTrigger.Text,
      triggerConfidence: primaryTrigger.GroupScore,
      raw: event,
    };

    for (const arg of event.Arguments) {
      if (arg.Score < ROLE_THRESHOLD) continue;  // Low-confidence role — skip rather than misassign
      const entity = entityMap.get(arg.EntityId);
      if (!entity) continue;

      switch (arg.Role) {
        case 'SUBJECT':    structured.subject   = entity.Text; break;
        case 'OBJECT':     structured.object    = entity.Text; break;
        case 'DATE_TIME':  structured.date      = entity.Text; break;
        case 'ARTIFACT':   structured.artifact  = entity.Text; break;
        case 'INVESTOR':   structured.investor  = entity.Text; break;
        case 'RECIPIENT':  structured.recipient = entity.Text; break;
      }
    }

    results.push(structured);
  }

  return results;
}

// Deduplicate ACQUISITION + CORPORATE_ACQUISITION with overlapping triggers
function deduplicateByTriggerOffset(events: StructuredEvent[]): StructuredEvent[] {
  const seen = new Set<string>();
  return events.filter(e => {
    const trigger = e.raw.Triggers[0];
    if (!trigger) return true;
    const key = `${trigger.BeginOffset}-${trigger.EndOffset}`;
    if (seen.has(key)) return false;
    seen.add(key);
    return true;
  });
}

Two-tool MCP pattern for async events extraction

import { S3Client, GetObjectCommand, PutObjectCommand } from '@aws-sdk/client-s3';
import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
import { z } from 'zod';
import { Readable } from 'node:stream';
import * as readline from 'node:readline';

const server = new McpServer({ name: 'financial-events', version: '1.0.0' });
const s3 = new S3Client({ region: 'us-east-1' });

// Tool 1: submit documents for events extraction — returns jobId immediately
server.tool('extract_events_start', {
  document_texts: z.array(z.string()).min(1).max(1000),
  event_types: z.array(z.enum([
    'ACQUISITION', 'CORPORATE_ACQUISITION', 'EMPLOYMENT', 'IPO',
    'BOARD_CHANGE', 'BANKRUPTCY', 'BUSINESS_EXPANSION', 'BUSINESS_REDUCTION',
    'DISSOLVED', 'RIGHTS_ISSUE', 'SECONDARY_OFFERING',
  ])).default(['ACQUISITION', 'CORPORATE_ACQUISITION', 'EMPLOYMENT', 'BOARD_CHANGE']),
}, async ({ document_texts, event_types }) => {
  const inputKey = `events-input/${Date.now()}.txt`;

  // Write documents to S3 as newline-separated file
  await s3.send(new PutObjectCommand({
    Bucket: process.env.COMPREHEND_BUCKET!,
    Key: inputKey,
    Body: document_texts.join('\n'),
    ContentType: 'text/plain',
  }));

  const { JobId } = await client.send(new StartEventsDetectionJobCommand({
    JobName: `events-${Date.now()}`,
    DataAccessRoleArn: process.env.COMPREHEND_ROLE_ARN!,
    LanguageCode: LanguageCode.EN,
    TargetEventTypes: event_types,
    InputDataConfig: {
      S3Uri: `s3://${process.env.COMPREHEND_BUCKET!}/${inputKey}`,
      InputFormat: 'ONE_DOC_PER_LINE',
    },
    OutputDataConfig: {
      S3Uri: `s3://${process.env.COMPREHEND_BUCKET!}/events-output/`,
    },
  }));

  return {
    content: [{
      type: 'text',
      text: JSON.stringify({
        jobId: JobId,
        status: 'submitted',
        document_count: document_texts.length,
        message: 'Call extract_events_results with this jobId to retrieve results (typically ready in 2–10 minutes)',
      }),
    }],
  };
});

// Tool 2: check status and retrieve structured events when ready
server.tool('extract_events_results', {
  job_id: z.string(),
}, async ({ job_id }) => {
  const { EventsDetectionJobProperties: props } = await client.send(
    new DescribeEventsDetectionJobCommand({ JobId: job_id })
  );

  const status = props?.JobStatus;

  if (status !== 'COMPLETED') {
    return {
      content: [{
        type: 'text',
        text: JSON.stringify({
          jobId: job_id,
          status: status ?? 'UNKNOWN',
          message: status === 'FAILED'
            ? `Job failed: ${props?.Message ?? 'no message'}`
            : 'Job not yet complete — retry in 30–60 seconds',
        }),
      }],
    };
  }

  // Read and parse output JSONL from S3
  const outputUri = props.OutputDataConfig?.S3Uri!;
  const [bucket, ...keyParts] = outputUri.replace('s3://', '').split('/');
  const key = `${keyParts.join('/')}/output`;

  const s3Obj = await s3.send(new GetObjectCommand({ Bucket: bucket, Key: key }));
  const stream = s3Obj.Body as Readable;

  const allEvents: StructuredEvent[] = [];
  const rl = readline.createInterface({ input: stream });

  for await (const line of rl) {
    if (!line.trim()) continue;
    const doc = JSON.parse(line) as DocumentEventsOutput;
    const docEvents = deduplicateByTriggerOffset(parseDocumentEvents(doc));
    allEvents.push(...docEvents);
  }

  return {
    content: [{
      type: 'text',
      text: JSON.stringify({
        jobId: job_id,
        status: 'COMPLETED',
        event_count: allEvents.length,
        events: allEvents.map(e => ({
          type: e.type,
          trigger: e.trigger,
          confidence: e.triggerConfidence,
          subject: e.subject,
          object: e.object,
          date: e.date,
          artifact: e.artifact,
        })),
      }),
    }],
  };
});

The two-tool pattern is necessary because Events detection is async-only and jobs take 2–15 minutes depending on document volume. Blocking inside a single tool handler for 10 minutes is impractical — it holds the MCP server's tool call thread, may exceed orchestrator timeouts, and prevents the AI agent from doing other work while waiting. The submit/retrieve split lets the orchestrator poll at its own pace and parallelize other tasks. The complete Events API reference including all 11 event type role vocabularies, output deduplication, and async job lifecycle management is in AWS Comprehend Events Detection in MCP Servers.

Production checklist: Comprehend Advanced NLP stack

AreaCheckWhy it matters
PII gateContainsPiiEntities called before any log write or external API call for user-supplied textAWS credential leaks to CloudWatch are a common accidental exposure vector
PII byte limitInput split into ≤4,500 UTF-8 byte chunks before calling sync PII APIs5,000 byte hard limit — silently returns empty Labels if exceeded
Redaction directionPII spans processed right-to-left (sorted descending by BeginOffset)Left-to-right replacement corrupts all downstream offsets after the first substitution
Comprehend IAM roleTrust policy has aws:SourceAccount condition for comprehend.amazonaws.comPrevents confused deputy attacks on the data access role
Entity type namesEntity types match [A-Z_]+ regex onlyDigits or hyphens in type names cause CreateEntityRecognizer to fail
Annotation volume≥200 annotated examples per entity type for production useModels trained with fewer examples often achieve F1 below 0.70 threshold
F1 gateEvaluationMetrics.F1Score >= 0.80 before deploying endpoint to productionBelow 0.70 F1 is near-random for many domains — check per-type breakdown first
Offset precisionAnnotation spans verified: line.slice(begin, end) === entityTextSystematic off-by-one errors are the most common cause of low F1 scores
Classification thresholdRoute to fallback handler when Classes[0].Score < 0.70Low-confidence routing is worse than no routing — creates silent misclassification
Events deduplicationACQUISITION + CORPORATE_ACQUISITION deduplicated by trigger BeginOffset/EndOffsetBoth types may fire for the same sentence, producing duplicate structured events
Events thresholdsTrigger GroupScore >= 0.70 and role Score >= 0.75 applied independentlyHigh trigger confidence with low role score = event happened but participant unclear — don't assign wrong entity
Async patternEvents tools use submit/retrieve split — no blocking for job completion inside tool handlerEvents jobs take 2–15 min — single blocking tool would exceed orchestrator timeouts