OpenSearch k-NN Vector Search for MCP Servers

k-NN engine selection: nmslib vs lucene vs faiss

OpenSearch supports three k-NN engines, each with different trade-offs for MCP workloads:

For AliveMCP's use case — semantic search across ~50,000 MCP endpoint embeddings — lucene is the correct engine. It supports native filtering (e.g., "status": "degraded" filtered similarity search) without post-filter result loss, and 50,000 vectors at 1536 dimensions (~300 MB) fits comfortably in a single shard's on-heap graph.

Index mapping: knn_vector field type

k-NN indexing is opt-in per index. Both "index.knn": true in settings and "type": "knn_vector" in the field mapping are required. Omitting the index setting stores the vector as a binary blob but builds no HNSW graph — queries silently return zero results.

// Create k-NN index for MCP endpoint embeddings
await client.indices.create({
  index: 'mcp-endpoints-v1',
  body: {
    settings: {
      'index.knn': true,                  // Required — enables k-NN graph construction
      'index.knn.algo_param.ef_search': 512,  // Query-time graph traversal depth (default 512)
      number_of_shards: 2,
      number_of_replicas: 1,
    },
    mappings: {
      properties: {
        server_slug:   { type: 'keyword' },
        display_name:  { type: 'text', analyzer: 'standard' },
        registry:      { type: 'keyword' },           // mcp.so, glama, smithery, etc.
        status:        { type: 'keyword' },           // live, dead, degraded
        checked_at:    { type: 'date' },
        // Vector field — must match your embedding model's output dimension
        description_embedding: {
          type: 'knn_vector',
          dimension: 1536,                            // OpenAI ada-002 / text-embedding-3-small
          method: {
            name: 'hnsw',
            engine: 'lucene',                         // lucene engine for filtered queries
            space_type: 'cosinesimil',                // cosine similarity for text embeddings
            parameters: {
              m: 16,           // Graph connectivity (16–64; higher = better recall, more memory)
              ef_construction: 256,  // Build quality (128–512; default 512)
            },
          },
        },
      },
    },
  },
});

The dimension parameter is immutable after index creation — it must exactly match the vector length your embedding model produces. Common dimensions: 1536 for OpenAI text-embedding-ada-002 and text-embedding-3-small, 3072 for text-embedding-3-large, 768 for many open-source models (BGE-base, E5-base), 384 for all-MiniLM-L6-v2.

HNSW parameters: m and ef_construction

The two critical HNSW build parameters are m (graph connectivity) and ef_construction (build-time traversal depth). A third parameter, ef_search, controls query recall and can be changed at query time without re-indexing.

For 50,000 MCP endpoint embeddings at 1536 dimensions with m=16: memory overhead is approximately 50,000 × 16 × 8 bytes × 2 layers ≈ 12.8 MB for the HNSW graph, plus 50,000 × 1536 × 4 bytes ≈ 307 MB for the raw vectors — well within a single shard's JVM heap allocation on an r6g.large.search instance (32 GB RAM, 50% heap).

Bulk indexing vectors

Vectors are indexed via the standard Bulk API — they are just JSON arrays in the document body. The only constraint is that the array length must equal the mapping's dimension value.

import Anthropic from '@anthropic-ai/sdk';
import { Client } from '@opensearch-project/opensearch';
import { AwsSigv4Signer } from '@opensearch-project/opensearch/aws';
import { defaultProvider } from '@aws-sdk/credential-provider-node';

const openai = new OpenAI();  // or Bedrock / Anthropic for embeddings
const os = new Client({
  ...AwsSigv4Signer({
    region: process.env.AWS_REGION ?? 'us-east-1',
    service: 'es',
    getCredentials: () => defaultProvider()(),
  }),
  node: `https://${process.env.OPENSEARCH_DOMAIN_ENDPOINT}`,
});

async function indexMcpEndpoints(endpoints: McpEndpoint[]): Promise {
  const BATCH = 50;  // Embed in batches of 50 to stay within rate limits

  for (let i = 0; i < endpoints.length; i += BATCH) {
    const batch = endpoints.slice(i, i + BATCH);

    // Generate embeddings for the batch
    const embeddingResponse = await openai.embeddings.create({
      model: 'text-embedding-3-small',
      input: batch.map(e => `${e.displayName}: ${e.description}`),
    });

    // Build bulk body: [action, document, action, document, ...]
    const bulkBody: unknown[] = [];
    for (let j = 0; j < batch.length; j++) {
      const endpoint = batch[j];
      const embedding = embeddingResponse.data[j].embedding;  // number[] of length 1536

      bulkBody.push({ index: { _index: 'mcp-endpoints-v1', _id: endpoint.slug } });
      bulkBody.push({
        server_slug: endpoint.slug,
        display_name: endpoint.displayName,
        registry: endpoint.registry,
        status: endpoint.status,
        checked_at: new Date().toISOString(),
        description_embedding: embedding,
      });
    }

    const response = await os.bulk({ body: bulkBody });

    // Always check body.errors — bulk returns HTTP 200 even on partial failure
    if (response.body.errors) {
      const failed = response.body.items
        .filter((item: any) => item.index?.error)
        .map((item: any) => ({ id: item.index._id, reason: item.index.error.reason }));
      console.error('Bulk index partial failure:', failed);
    }
  }
}

k-NN query with filter support (lucene engine)

The lucene engine is the only OpenSearch engine that supports pre-filtering via the filter parameter inside the knn query. Pre-filtering applies the boolean filter before graph traversal, so all k results respect the filter. Post-filter engines (nmslib, faiss-HNSW) apply the filter after retrieving k candidates, which means you can get fewer than k results when the filter is selective.

async function findSimilarDegradedEndpoints(
  queryEmbedding: number[],
  k = 10,
): Promise {
  const response = await os.search({
    index: 'mcp-endpoints-v1',
    body: {
      size: k,
      query: {
        knn: {
          description_embedding: {
            vector: queryEmbedding,
            k,
            // lucene engine: pre-filter applied before HNSW traversal
            filter: {
              bool: {
                must: [
                  { term: { status: 'degraded' } },
                ],
              },
            },
          },
        },
      },
      _source: ['server_slug', 'display_name', 'registry', 'status', 'checked_at'],
    },
  });

  return response.body.hits.hits.map((hit: any) => ({
    ...hit._source,
    score: hit._score,
  }));
}

For post-filter engines (nmslib, faiss), multiply k by the inverse selectivity of your filter to compensate. If the filter passes 10% of documents, set k = desired_results × 10 to ensure enough candidates survive the post-filter. This overprovisioning increases latency proportionally.

Hybrid BM25 + k-NN search

Pure k-NN queries miss exact keyword matches (e.g., searching "PostgreSQL MCP server" by embedding may rank Redis servers highly if their descriptions share semantic overlap). Hybrid search combines a BM25 text score with a k-NN vector score using a hybrid query type (available in OpenSearch 2.10+).

// Hybrid search: combine BM25 text match with k-NN vector similarity
const hybridResponse = await os.search({
  index: 'mcp-endpoints-v1',
  body: {
    query: {
      hybrid: {
        queries: [
          {
            match: {
              display_name: {
                query: searchText,
                boost: 1.0,
              },
            },
          },
          {
            knn: {
              description_embedding: {
                vector: queryEmbedding,
                k: 20,
              },
            },
          },
        ],
      },
    },
    // Normalization processor must be configured in the search pipeline
    search_pipeline: 'hybrid-search-pipeline',
    size: 10,
  },
});

The hybrid query requires a search pipeline with a normalization-processor (to put BM25 and k-NN scores on the same scale, typically min_max normalization) and a combination-processor (arithmetic mean or harmonic mean). Create the pipeline once via PUT /_search/pipeline/hybrid-search-pipeline.

k-NN plugin health probe

The k-NN plugin exposes a stats API that reveals graph build errors, cache hit rates, and evictions — useful for detecting degraded vector search performance before query latency spikes surface to users.

// k-NN plugin stats — check for graph build errors and cache health
const stats = await os.transport.request({
  method: 'GET',
  path: '/_plugins/_knn/stats',
});

// Key metrics to monitor:
// knn_query_requests: cumulative query count
// knn_query_errors: non-zero means index corruption or dimension mismatch
// cache_capacity_reached: true means HNSW graphs are being evicted (JVM heap pressure)
// graph_index_errors: non-zero means bulk indexing failures
const {
  knn_query_requests,
  knn_query_errors,
  cache_capacity_reached,
  graph_index_errors,
} = stats.body.nodes[Object.keys(stats.body.nodes)[0]];

if (knn_query_errors > 0 || cache_capacity_reached || graph_index_errors > 0) {
  console.error('k-NN plugin unhealthy:', { knn_query_errors, cache_capacity_reached, graph_index_errors });
}

cache_capacity_reached: true means the HNSW graph has been evicted from the node's circuit-breaker-controlled k-NN cache — subsequent queries fall back to brute-force exact search, which is orders of magnitude slower. Resolve by increasing the knn.memory.circuit_breaker.limit cluster setting (default 50% of JVM heap) or moving to a larger instance type.

How AliveMCP uses k-NN for endpoint similarity

AliveMCP embeds every MCP server's name and description using a 768-dimensional open-source model (BGE-base-en-v1.5 via a self-hosted inference endpoint) and stores the vectors in OpenSearch. When a new MCP server goes down, the system queries the k-NN index for the 5 most similar servers that are currently live — the similarity search uses the filter: { term: { status: "live" } } pre-filter — and surfaces those as "your server is down; these alternatives are healthy" links on the status page.

The vector index also powers the weekly "MCP servers in the same category that improved uptime this week" digest email sent to Author tier subscribers. A temporal filter (checked_at >= now-7d) combined with a k-NN query against the subscriber's claimed endpoint embedding produces a personalised similarity-ranked list of peer servers — entirely from the OpenSearch index, no separate recommendation service needed.

Related guides

• OpenSearch Overview for MCP Servers — bulk indexing, scroll, SigV4 auth
• OpenSearch Index State Management for Probe Time-Series Data
• OpenSearch Serverless Collections for MCP Tool Backends
• OpenSearch Anomaly Detection for MCP Endpoint Health