AWS OpenSearch Service Advanced Patterns for MCP Monitoring: Vector Search, ISM Lifecycle, Serverless Trade-offs, and Anomaly Detection

Pattern 1 — The Vector Intelligence Layer: k-NN Engine Selection, Index Mapping, Filtered Queries, and Hybrid Search

Choosing between nmslib, lucene, and faiss

OpenSearch's k-NN plugin supports three backend engines, each with different trade-offs for the size, filter requirements, and throughput of MCP endpoint embedding workloads.

For AliveMCP's endpoint similarity search — approximately 50,000 MCP server description embeddings at 768 dimensions — the lucene engine is the correct choice. It supports native pre-filtering (filter applied before HNSW graph traversal, not after), 50,000 vectors at 768 dimensions occupies roughly 150 MB on-heap, and the workload is read-heavy with low concurrent query volume. nmslib is deprecated on OpenSearch 2.13 and later; any index that still uses it silently falls back to lucene behaviour in queries but the deprecation warning fills CloudWatch logs with noise and the engine receives no further optimisations.

Index mapping: knn_vector field type and the index.knn requirement

k-NN indexing is opt-in at the index level. Both a settings flag and a field mapping type are required — omitting either one produces a vector field that stores data correctly but builds no HNSW graph, returning zero results silently on k-NN queries.

import { Client } from '@opensearch-project/opensearch';
import { AwsSigv4Signer } from '@opensearch-project/opensearch/aws';
import { defaultProvider } from '@aws-sdk/credential-provider-node';

const os = new Client({
  ...AwsSigv4Signer({
    region: process.env.AWS_REGION ?? 'us-east-1',
    service: 'es',  // 'es' for managed domain; 'aoss' for Serverless — see Pattern 2
    getCredentials: () => defaultProvider()(),
  }),
  node: `https://${process.env.OPENSEARCH_DOMAIN_ENDPOINT}`,
});

await os.indices.create({
  index: 'mcp-endpoints-v1',
  body: {
    settings: {
      'index.knn': true,                        // Required — without this, knn_vector fields are inert blobs
      'index.knn.algo_param.ef_search': 512,    // Query-time graph traversal depth (default 512)
      number_of_shards: 2,
      number_of_replicas: 1,
    },
    mappings: {
      properties: {
        server_slug:   { type: 'keyword' },
        display_name:  { type: 'text', analyzer: 'standard' },
        registry:      { type: 'keyword' },     // mcp.so, glama, smithery, pulsemcp, etc.
        status:        { type: 'keyword' },     // live, dead, degraded
        checked_at:    { type: 'date' },
        description_embedding: {
          type: 'knn_vector',
          dimension: 768,                       // Must match embedding model output exactly — immutable after creation
          method: {
            name: 'hnsw',
            engine: 'lucene',                   // lucene for filtered queries and <10M vectors
            space_type: 'cosinesimil',          // cosine similarity for text embeddings
            parameters: {
              m: 16,             // Graph connectivity: higher = better recall, more memory (8–64)
              ef_construction: 256,  // Build quality: higher = better graph, slower indexing (128–512)
            },
          },
        },
      },
    },
  },
});

The dimension parameter is immutable after index creation. Common values: 1536 for OpenAI text-embedding-ada-002 and text-embedding-3-small, 3072 for text-embedding-3-large, 768 for BGE-base-en-v1.5 and E5-base, 384 for all-MiniLM-L6-v2. AliveMCP uses 768-dimensional BGE-base embeddings run locally, which avoids the per-embedding API cost for a corpus that refreshes infrequently.

HNSW parameters: m, ef_construction, and ef_search

Three parameters govern HNSW recall quality and resource usage. Two are fixed at index creation; one can be tuned per-query without re-indexing.

For 50,000 endpoint embeddings at 768 dimensions with m=16: HNSW graph overhead is approximately 50,000 × 16 × 8 bytes × 2 layers ≈ 12.8 MB. Raw vector storage is 50,000 × 768 × 4 bytes ≈ 154 MB. Total on-heap footprint ~167 MB — well within a single shard on even the smallest search instance type.

Filtered k-NN queries: pre-filter vs post-filter

The most common k-NN mistake in MCP monitoring is using a post-filter engine (nmslib, faiss-HNSW) with a selective filter — for example, filtering for only "live" servers when 70% of monitored endpoints are dead or degraded. Post-filter retrieves k approximate nearest neighbours from the full corpus, then applies the filter, which discards most results and returns far fewer than k matches. With the lucene engine, the filter is applied before graph traversal — the HNSW search operates only within the matching subset, so all k results respect the filter regardless of selectivity.

// lucene engine: pre-filter — all k results are guaranteed to match the filter
async function findSimilarLiveEndpoints(
  queryEmbedding: number[],
  excludeSlug: string,
  k = 5,
): Promise {
  const response = await os.search({
    index: 'mcp-endpoints-v1',
    body: {
      size: k,
      query: {
        knn: {
          description_embedding: {
            vector: queryEmbedding,
            k,
            // lucene pre-filter: applied BEFORE graph traversal
            // With faiss/nmslib post-filter, you'd need k = desired_results / filter_selectivity
            filter: {
              bool: {
                must: [
                  { term: { status: 'live' } },
                ],
                must_not: [
                  { term: { server_slug: excludeSlug } },  // Exclude the server that went down
                ],
              },
            },
          },
        },
      },
      _source: ['server_slug', 'display_name', 'registry', 'status'],
    },
  });

  return response.body.hits.hits.map((hit: any) => ({
    ...hit._source,
    similarity: hit._score,
  }));
}

Hybrid BM25 + k-NN search

Pure k-NN queries miss exact keyword matches: searching for "PostgreSQL MCP server" by embedding may rank Redis or MySQL servers highly if their descriptions share semantic overlap around "database" and "query". Hybrid search combines a BM25 keyword score with a k-NN vector score using a hybrid query type (available in OpenSearch 2.10+), but the two score scales are incompatible — BM25 scores are unbounded floats while cosine similarity scores are in [0, 1]. A normalization processor in a search pipeline corrects this.

// One-time pipeline creation — run at deploy time
await os.transport.request({
  method: 'PUT',
  path: '/_search/pipeline/hybrid-search-pipeline',
  body: {
    description: 'Normalize and combine BM25 and k-NN scores for MCP endpoint search',
    phase_results_processors: [
      {
        'normalization-processor': {
          normalization: { technique: 'min_max' },
          combination: { technique: 'arithmetic_mean', parameters: { weights: [0.3, 0.7] } },
          // 0.3 weight for BM25, 0.7 for k-NN — tune based on search quality
        },
      },
    ],
  },
});

// Hybrid search query
const hybridResults = await os.search({
  index: 'mcp-endpoints-v1',
  body: {
    search_pipeline: 'hybrid-search-pipeline',
    query: {
      hybrid: {
        queries: [
          {
            match: {
              display_name: { query: searchText, boost: 1.0 },
            },
          },
          {
            knn: {
              description_embedding: {
                vector: queryEmbedding,
                k: 20,  // Retrieve more k-NN candidates; pipeline re-ranks after normalization
              },
            },
          },
        ],
      },
    },
    size: 10,
    _source: ['server_slug', 'display_name', 'registry', 'status'],
  },
});

k-NN plugin health monitoring

The k-NN stats API exposes graph build errors, query errors, and cache pressure — the three signals that indicate vector search is degraded without any query latency spike yet visible to users.

const stats = await os.transport.request({
  method: 'GET',
  path: '/_plugins/_knn/stats',
});

const nodeStats = stats.body.nodes[Object.keys(stats.body.nodes)[0]];
const {
  knn_query_errors,       // Non-zero: index corruption or dimension mismatch
  cache_capacity_reached, // True: HNSW graphs evicted from JVM heap — brute-force fallback active
  graph_index_errors,     // Non-zero: bulk indexing failures dropped vectors
} = nodeStats;

if (knn_query_errors > 0 || cache_capacity_reached || graph_index_errors > 0) {
  // Alert: vector search degraded; similarity queries may return incorrect results
  // cache_capacity_reached = true means k-NN is silently running brute-force (O(n) vs O(log n))
  // Resolve by increasing knn.memory.circuit_breaker.limit (default 50% JVM heap) or upsizing instance
}

cache_capacity_reached: true is the most dangerous silent failure mode in k-NN workloads. When the HNSW graph is evicted from the k-NN circuit-breaker-controlled cache, subsequent queries fall back to brute-force exhaustive search — orders of magnitude slower, but still returning results. From a caller's perspective, the k-NN feature appears to be working. From a latency perspective, p99 query time for 50,000 vectors goes from ~5ms to ~500ms. The only way to detect this without monitoring is noticing the latency spike. The stats API makes it explicit.

Pattern 2 — The Time-Series Lifecycle Model: ISM Hot-Warm-Cold, Rollover Aliases, and the Serverless Deployment Trade-off

ISM policy structure: states, actions, and transitions

An ISM policy defines a state machine applied to one or more indices. Each state has actions (operations to perform once while in that state) and transitions (conditions that move the index to the next state). The ISM worker checks each managed index every 5 minutes by default.

For AliveMCP's probe results — approximately 83 documents per second at 300,000 monitored endpoints — a single index reaches 50 GB in about 7 days. The lifecycle policy manages 13 rolling indices concurrently, totalling ~650 GB of data, with the oldest data on UltraWarm nodes at roughly 90% lower cost per GB than hot SSD.

const lifecyclePolicy = {
  policy: {
    description: 'AliveMCP probe results: 7d hot, 30d warm (UltraWarm), 60d cold, delete at 90d',
    default_state: 'hot',
    states: [
      {
        name: 'hot',
        actions: [
          {
            rollover: {
              min_doc_count: 50_000_000,  // 50M probe results (~7 days)
              min_size: '50gb',
              min_index_age: '7d',        // Whichever condition is met first triggers rollover
            },
          },
        ],
        transitions: [
          { state_name: 'warm', conditions: { min_rollover_age: '1d' } },
        ],
      },
      {
        name: 'warm',
        actions: [
          { replica_count: { number_of_replicas: 0 } }, // Cut storage cost before merge
          { force_merge: { max_num_segments: 1 } },      // 20-40% storage reduction; takes 10-40min
          { allocation: { require: { box_type: 'warm' } } }, // Move to UltraWarm nodes (S3-backed)
          // WARNING: allocation to warm requires UltraWarm nodes in the cluster.
          // Without them, shards enter INITIALIZING and force_merge never runs.
        ],
        transitions: [
          { state_name: 'cold', conditions: { min_index_age: '30d' } },
        ],
      },
      {
        name: 'cold',
        actions: [
          { read_only: {} },
          { allocation: { require: { box_type: 'cold' } } },
        ],
        transitions: [
          { state_name: 'delete', conditions: { min_index_age: '90d' } },
        ],
      },
      {
        name: 'delete',
        actions: [{ delete: {} }],
        transitions: [],
      },
    ],
    ism_template: [
      {
        index_patterns: ['probe-results-*'],
        priority: 100,  // Higher priority wins when multiple templates match an index
      },
    ],
  },
};

The rollover alias pattern: why writing directly to the index breaks ISM

ISM's rollover action only fires when writes go through an alias with "is_write_index": true. Writing directly to the backing index name — even the current one — bypasses ISM's rollover trigger entirely. The index accumulates without limit while ISM's other state transitions (warm, cold, delete) still run on the index at its existing age, creating the paradox of data being deleted at 90 days while the index itself never rolled over and therefore contains mixed-age data spanning multiple rollover periods.

// Create the initial backing index with write and read aliases
await os.indices.create({
  index: 'probe-results-000001',
  body: {
    aliases: {
      'probe-results-write': { is_write_index: true },   // ISM rollover fires on this alias
      'probe-results-read':  {},                          // Read queries span all backing indices
    },
    settings: {
      'index.plugins.index_state_management.policy_id': 'probe-results-lifecycle',
    },
  },
});

// Correct: write via the alias — ISM rollover triggers when min_doc_count/min_size/min_age met
await os.index({
  index: 'probe-results-write',
  body: { server_slug, status, latency_ms, checked_at: new Date() },
});

// Wrong: direct index write — rollover never fires, index grows unbounded
// await os.index({ index: 'probe-results-000001', body: probeResult });

// After rollover: backing sequence becomes probe-results-000001, -000002, -000003...
// probe-results-read alias automatically spans all backing indices for historical queries

ISM templates: zero-maintenance policy attachment

Without an ISM template, each new rollover index (probe-results-000002 through probe-results-000099) is created without a policy attached. The ISM worker ignores it; data accumulates without lifecycle management. The ism_template array in the policy definition auto-attaches the policy to any index whose name matches the pattern at creation time — set it once, and every rollover index inherits the lifecycle policy automatically.

To verify a new index received the template-attached policy, check its settings after creation:

const settings = await os.indices.getSettings({ index: 'probe-results-000002' });
const policyId = settings.body['probe-results-000002']?.settings?.index
  ?.plugins?.index_state_management?.policy_id;

if (!policyId) {
  // Template did not attach — check that ism_template index_patterns match the index name
  // and that the ISM template priority is high enough to win over other matching templates
  console.error('probe-results-000002 has no ISM policy attached');
}

Forced merge: I/O impact and scheduling strategy

Forced merge rewrites all Lucene segments in a shard into a single segment, reducing storage 20–40% and cutting query time for aggregation-heavy analytics (90-day histograms drop from ~800ms with 200 segments to ~120ms with 1 merged segment). The operation is disk I/O saturating and takes 10–40 minutes for a 50 GB index.

ISM executes actions within a state sequentially in definition order. Placing replica_count and allocation before force_merge in the warm state ensures shard count is reduced and the UltraWarm migration has started before the merge begins. For production environments with strict p99 SLAs, split the warm state into two: warm-setup for replica reduction and allocation, and warm-merge triggered min_index_age: 2h after warm-setup — this constrains forced merge to run at least 2 hours after the last write, outside the probe peak window.

OpenSearch Serverless: when to use it and when not to

Amazon OpenSearch Serverless removes cluster sizing and scaling decisions. You create a collection, apply three policy layers, and write documents. The SigV4 service name is 'aoss', not 'es' — this single character difference is the most common Serverless integration failure.

// OpenSearch Serverless client — 'aoss' not 'es'
import { Client } from '@opensearch-project/opensearch';
import { AwsSigv4Signer } from '@opensearch-project/opensearch/aws';
import { defaultProvider } from '@aws-sdk/credential-provider-node';

const serverlessClient = new Client({
  ...AwsSigv4Signer({
    region: process.env.AWS_REGION ?? 'us-east-1',
    service: 'aoss',  // 'aoss' for Serverless — using 'es' returns HTTP 403 with no error body
    getCredentials: () => defaultProvider()(),
  }),
  node: `https://${process.env.AOSS_COLLECTION_ENDPOINT}`,
  // Endpoint format: {collection-id}.{region}.aoss.amazonaws.com
});

The cost floor for Serverless is 2 indexing OCUs + 2 search OCUs at $0.24/OCU-hour = approximately $0.48/hour = $350/month even when idle. A managed t3.medium.search domain costs ~$60/month. Serverless is only cheaper when managed domain auto-scaling to handle burst queries would push the cost above $350/month — roughly at 50,000+ concurrent queries per hour with unpredictable bursts. At AliveMCP's scale — 83 writes/second, moderate query volume — the managed domain is definitively cheaper.

The feature gap is the more important constraint for most MCP monitoring teams. Serverless does not support ISM policies, the scroll API, cat APIs, cluster admin APIs, the anomaly detection plugin, or the alerting plugin. All four advanced features in this guide require a managed domain.

Pattern 3 — Anomaly Detection as Early-Warning: HC Detectors, RCF Warm-up, Alerting Integration, and Entity Scaling

Why binary thresholds miss the earliest signal

Binary up/down monitoring answers "did the probe succeed?" — it fires when a server is already failing. In practice, MCP server failures are rarely instantaneous. They are preceded by latency creep: p95 response time climbing from 50ms to 200ms to 800ms over 3–15 minutes before the first timeout. Binary monitoring misses this entire window. OpenSearch Anomaly Detection scores each time window against an individually trained baseline per server, catching the "this server is behaviorally unusual" signal before the "this server is timing out" threshold fires.

HC detector creation: one RCF model per server slug

A High Cardinality (HC) detector uses a category_field (a keyword field) to partition the training data — one RCF model sub-detector runs per unique value of that field. For MCP endpoint monitoring, category_field: "server_slug" gives each server its own baseline. A server that always takes 800ms does not alert; a server that normally takes 50ms but suddenly hits 400ms does.

// Create HC anomaly detector for per-server latency anomalies
const detectorResponse = await os.transport.request({
  method: 'POST',
  path: '/_plugins/_anomaly_detection/detectors',
  body: {
    name: 'mcp-endpoint-latency-hc',
    description: 'Per-server-slug p95 latency anomaly detection for MCP endpoints',
    time_field: 'checked_at',
    indices: ['probe-results-*'],          // All probe-results rollover indices
    feature_attributes: [
      {
        feature_name: 'p95_latency_ms',
        feature_enabled: true,
        aggregation_query: {
          p95_latency_ms: {
            percentiles: { field: 'latency_ms', percents: [95] },
          },
        },
      },
    ],
    filter_query: {
      bool: {
        must: [
          { term: { probe_type: 'http_check' } },
          { term: { maintenance_mode: false } },  // Exclude servers in maintenance
        ],
      },
    },
    detector_interval: {
      period: { interval: 1, unit: 'Minutes' },  // 1m — critical: default 10m is too coarse for 60s probes
    },
    window_delay: {
      period: { interval: 30, unit: 'Seconds' }, // Wait 30s for late-arriving probe data
    },
    shingle_size: 8,             // RCF context window: 8 × 1m = 8 minutes of history
    category_field: ['server_slug'],  // HC: one sub-detector per unique server_slug
  },
});

const detectorId = detectorResponse.body._id;

RCF algorithm: shingle size and the warm-up window

The Random Cut Forest algorithm builds a forest of random binary trees over a sliding window of shingle_size consecutive data points. Each point in the shingle includes all feature values at that interval, giving the model a multi-dimensional view of recent history. A new data point is anomalous if adding it to the forest increases its complexity more than expected given the trained baseline.

Warm-up: the RCF forest requires at least shingle_size × interval data points (8 minutes at shingle_size=8, interval=1m) before producing reliable scores. During warm-up, the detector sets anomaly_grade: 0 for all intervals regardless of actual latency. The same warm-up restarts after any data gap exceeding 8 minutes — a server that stops receiving probes for 10 minutes and then resumes will have 8 minutes of suppressed anomaly detection. Never alert on results where both anomaly_grade == 0 and confidence == 0.

Preview mode: validate feature extraction before starting real-time detection

Preview mode runs the detector against historical data without writing to the anomaly results index. Use it before starting the detector to confirm that the feature aggregation returns non-null values and that the anomaly rate on normal data is in the 0.5–5% range.

const preview = await os.transport.request({
  method: 'POST',
  path: `/_plugins/_anomaly_detection/detectors/${detectorId}/_preview`,
  body: {
    period_start: Date.now() - 7 * 24 * 60 * 60 * 1000,
    period_end: Date.now(),
  },
});

const sampleResults = preview.body.anomaly_result.slice(0, 5);
for (const result of sampleResults) {
  if (result.feature_data[0].data === null) {
    // Feature returned null: check field name ('latency_ms' exists in probe-results-* ?)
    // and that the index_patterns in the detector match at least one existing index
    console.warn('Feature null — verify field name and index pattern');
  }
}

const anomalyRate = preview.body.anomaly_result.filter(r => r.anomaly_grade > 0).length
  / preview.body.anomaly_result.length;

// 0.5–5% is healthy; >10% means the feature is too noisy (use a different aggregation or longer interval)
// 0% means the feature has no variance (check data quality — are all latency_ms values identical?)
console.log(`Preview anomaly rate: ${(anomalyRate * 100).toFixed(1)}%`);

Reading anomaly results and starting real-time detection

// Start real-time detection
await os.transport.request({
  method: 'POST',
  path: `/_plugins/_anomaly_detection/detectors/${detectorId}/_start`,
});

// Query anomaly results for a specific server in the last hour
const anomalyResults = await os.search({
  index: '.opendistro-anomaly-results-history-*',  // System index for anomaly results
  body: {
    query: {
      bool: {
        must: [
          { term: { detector_id: detectorId } },
          { term: { 'entity.0.value': 'my-mcp-server-slug' } },  // HC entity filter
          { range: { data_start_time: { gte: 'now-1h' } } },
          { range: { anomaly_grade: { gt: 0.5 } } },            // Only significant anomalies
        ],
      },
    },
    sort: [{ data_start_time: 'desc' }],
    _source: ['data_start_time', 'anomaly_grade', 'confidence', 'feature_data', 'entity'],
  },
});

Alerting integration: wiring the detector to a webhook destination

// Create alerting monitor that fires when anomaly_grade > 0.7 and confidence > 0.5
await os.transport.request({
  method: 'POST',
  path: '/_plugins/_alerting/monitors',
  body: {
    type: 'monitor',
    name: 'mcp-endpoint-latency-anomaly-alert',
    monitor_type: 'anomaly_detector_monitor',
    enabled: true,
    schedule: { period: { interval: 1, unit: 'MINUTES' } },
    inputs: [{ anomaly_detector: { detector_id: detectorId } }],
    triggers: [
      {
        anomaly_detector_trigger: {
          name: 'high-anomaly-grade',
          severity: '2',
          min_anomaly_grade: 0.7,   // Only fire for high-confidence anomalies
          min_confidence: 0.5,      // Suppresses warm-up period (confidence is 0 during warm-up)
        },
      },
    ],
    actions: [
      {
        name: 'notify-alivemcp-webhook',
        destination_id: 'WEBHOOK_DESTINATION_ID',
        message_template: {
          source: `{
            "text": "MCP Endpoint Degraded: {{ctx.results[0].hits.hits[0]._source.entity[0].value}} — grade={{ctx.results[0].hits.hits[0]._source.anomaly_grade}} confidence={{ctx.results[0].hits.hits[0]._source.confidence}}"
          }`,
        },
        action_execution_policy: {
          action_execution_scope: 'PER_ALERT',  // One notification per unique anomalous server
        },
      },
    ],
  },
});

HC entity limit: scaling beyond 1,000 server slugs

Each HC detector tracks at most 1,000 unique entity values. When more than 1,000 unique server slugs appear in the training data, the detector's priority queue retains the 1,000 most frequently observed entities — new or infrequently probed servers are silently excluded from anomaly detection. No error is raised; the excluded servers simply never have anomaly scores computed.

To monitor more than 1,000 MCP servers with per-entity detection, partition across multiple HC detectors. AliveMCP's strategy: one HC detector per major registry source (MCP.so, Glama, Smithery, PulseMCP), each covering approximately 500 servers, with a global single-metric detector monitoring fleet-wide probe pipeline anomalies (total probe throughput, aggregate p95 latency across all servers) as a backstop.

// Check how many entity models are active in an HC detector
const entityProfile = await os.transport.request({
  method: 'GET',
  path: `/_plugins/_anomaly_detection/detectors/${detectorId}/profile/models`,
});

const activeEntities = entityProfile.body.entities?.length ?? 0;
console.log(`Active entity models: ${activeEntities} / 1000 max`);
// If approaching 1000, provision a second HC detector for the overflow registry

Failure modes reference

Related guides

• OpenSearch k-NN Vector Search for Semantic MCP Endpoint Discovery
• OpenSearch Index State Management for Probe Time-Series Data
• Amazon OpenSearch Serverless — collection types, access policies, OCU model
• OpenSearch Anomaly Detection for MCP Endpoint Health
• OpenSearch Overview for MCP Servers — bulk indexing, scroll, SigV4 auth
• How to Monitor an MCP Server
• MCP Server Uptime Monitoring
• AWS MSK for MCP Server Probe Pipelines: Event Architecture, IAM Auth, and Operational Reliability
• AWS MemoryDB for MCP Servers: Durable State, ACL Security, and Cost-Efficient Probe Archives