Bedrock Embeddings in MCP Servers — Titan V2, Cohere Embed, semantic search

Titan Text Embeddings V2 — API and dimensions

Amazon Titan Text Embeddings V2 (amazon.titan-embed-text-v2:0) produces dense vector embeddings for English and multilingual text. It offers three output dimension sizes: 256, 512, and 1024. The 1024-dimension model provides the highest retrieval quality and is the correct choice for production semantic search unless you have strict storage or memory constraints. 256 dimensions reduces storage by 4× at a 10–15% quality degradation on most benchmarks.

import {
  BedrockRuntimeClient,
  InvokeModelCommand,
} from '@aws-sdk/client-bedrock-runtime';

const client = new BedrockRuntimeClient({ region: 'us-east-1' });

// Required IAM permission: bedrock:InvokeModel on
// arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-embed-text-v2:0

interface TitanEmbedResponse {
  embedding: number[];
  inputTextTokenCount: number;
}

async function embedWithTitan(
  text: string,
  dimensions: 256 | 512 | 1024 = 1024,
  normalize = true,
): Promise {
  const body = JSON.stringify({
    inputText: text,
    dimensions,
    normalize,   // true = unit-normalized vectors; dot product = cosine similarity
  });

  const response = await client.send(new InvokeModelCommand({
    modelId: 'amazon.titan-embed-text-v2:0',
    contentType: 'application/json',
    accept: 'application/json',
    body: new TextEncoder().encode(body),
  }));

  const decoded = new TextDecoder().decode(response.body);
  const result: TitanEmbedResponse = JSON.parse(decoded);

  // Log token count to monitor chunking effectiveness
  console.debug(`Titan embed: ${result.inputTextTokenCount} tokens → ${dimensions}-dim vector`);

  return result.embedding;
}

// Cosine similarity (dot product of unit vectors)
function cosineSimilarity(a: number[], b: number[]): number {
  // When normalize=true on embedding, this is exact cosine similarity
  return a.reduce((sum, val, i) => sum + val * b[i], 0);
}

// Example: find most similar MCP server description
async function findSimilarServer(
  queryText: string,
  serverDescriptions: Array<{ slug: string; description: string }>,
): Promise {
  const [queryVec, ...docVecs] = await Promise.all([
    embedWithTitan(queryText),
    ...serverDescriptions.map(s => embedWithTitan(s.description)),
  ]);

  let bestIdx = 0;
  let bestScore = -1;
  for (let i = 0; i < docVecs.length; i++) {
    const score = cosineSimilarity(queryVec, docVecs[i]);
    if (score > bestScore) { bestScore = score; bestIdx = i; }
  }

  return serverDescriptions[bestIdx].slug;
}

The normalize=true parameter (default: true) returns unit-length vectors. With normalized vectors, cosine similarity reduces to a simple dot product — no need to divide by vector magnitudes. Always use normalize=true unless you specifically need raw unnormalized vectors for L2-distance comparisons.

Titan V2 supports 8192 tokens. For reference, 8192 tokens is approximately 32,000 characters or a 24-page document. Most MCP tool inputs are well within this limit; the constraint matters for MCP servers that embed full document contents rather than summaries.

Cohere Embed 3 — asymmetric search and quantization

Cohere Embed 3 (cohere.embed-english-v3 for English, cohere.embed-multilingual-v3 for 100+ languages) differs from Titan in two important ways: it uses asymmetric embeddings via the input_type parameter, and it supports quantized output types (int8, binary) for storage compression.

Asymmetric search means the model is trained to embed queries differently from documents — a query embedding "what is the AliveMCP uptime check interval?" is optimized to retrieve a document embedding "AliveMCP pings MCP endpoints every 60 seconds to detect downtime." Using the wrong input_type (embedding both query and documents as search_query or both as search_document) degrades retrieval quality by 5–15% on typical RAG benchmarks.

type CohereInputType =
  | 'search_query'       // short user queries
  | 'search_document'    // documents/passages to index
  | 'classification'     // text classification (no asymmetry needed)
  | 'clustering';        // grouping similar items

type CohereEmbeddingType =
  | 'float'              // 32-bit float, 1024 dims — full quality
  | 'int8'               // 8-bit integer, 1024 dims — 4× smaller, ~1% quality loss
  | 'binary';            // 1-bit packed, 128 bytes — 32× smaller, ~5–10% quality loss

interface CohereEmbedRequest {
  texts: string[];
  input_type: CohereInputType;
  embedding_types?: CohereEmbeddingType[];
  truncate?: 'NONE' | 'START' | 'END';  // default NONE — error on overlong input
}

interface CohereEmbedResponse {
  id: string;
  embeddings: {
    float?: number[][];
    int8?: number[][];
    binary?: number[][];
  };
  texts: string[];
  response_type: 'embeddings_floats' | 'embeddings_bytes';
}

async function embedWithCohere(
  texts: string[],
  inputType: CohereInputType,
  embeddingType: CohereEmbeddingType = 'float',
): Promise {
  const body = JSON.stringify({
    texts,
    input_type: inputType,
    embedding_types: [embeddingType],
    truncate: 'END',   // truncate to 512 tokens if over limit (safer than NONE)
  } satisfies CohereEmbedRequest);

  const response = await client.send(new InvokeModelCommand({
    modelId: 'cohere.embed-english-v3',
    contentType: 'application/json',
    accept: 'application/json',
    body: new TextEncoder().encode(body),
  }));

  const decoded = new TextDecoder().decode(response.body);
  const result: CohereEmbedResponse = JSON.parse(decoded);

  const embeddings = result.embeddings[embeddingType];
  if (!embeddings) throw new Error(`No ${embeddingType} embeddings in response`);
  return embeddings;
}

// Correct asymmetric usage pattern
async function buildSearchIndex(documents: string[]): Promise {
  return embedWithCohere(documents, 'search_document');
}

async function embedQuery(query: string): Promise {
  const vectors = await embedWithCohere([query], 'search_query');
  return vectors[0];
}

Cohere accepts up to 96 texts per API call (batch embedding). For an MCP server indexing a large corpus, batch the inputs to minimize API call overhead. Note that the per-text token limit is 512 tokens — significantly lower than Titan's 8192. For longer documents, chunk at paragraph or sentence boundaries before embedding.

Chunking strategy for MCP tool documents

When an MCP tool needs to embed documents that exceed the embedding model's token limit — or when semantic retrieval quality would benefit from more granular matching — chunking breaks each document into smaller overlapping passages before embedding.

interface Chunk {
  text: string;
  docId: string;
  chunkIndex: number;
  startChar: number;
}

function chunkDocument(
  text: string,
  docId: string,
  targetTokens = 400,   // ~300 chars per token → ~120k chars target
  overlapTokens = 50,
): Chunk[] {
  const approxCharsPerToken = 4;
  const chunkSize = targetTokens * approxCharsPerToken;
  const overlapSize = overlapTokens * approxCharsPerToken;

  const chunks: Chunk[] = [];
  let start = 0;

  while (start < text.length) {
    // Find a sentence boundary near chunkSize
    let end = Math.min(start + chunkSize, text.length);
    if (end < text.length) {
      const sentenceEnd = text.lastIndexOf('. ', end);
      if (sentenceEnd > start + chunkSize * 0.5) end = sentenceEnd + 2;
    }

    chunks.push({
      text: text.slice(start, end),
      docId,
      chunkIndex: chunks.length,
      startChar: start,
    });

    start = end - overlapSize;
    if (start <= 0) break;
  }

  return chunks;
}

// Build a full embedding index for a corpus of MCP tool documentation
async function buildCorpusIndex(
  docs: Array<{ id: string; content: string }>,
): Promise> {
  const allChunks = docs.flatMap(doc =>
    chunkDocument(doc.content, doc.id)
  );

  // Embed in batches of 96 (Cohere limit)
  const results: Array<{ chunk: Chunk; embedding: number[] }> = [];
  for (let i = 0; i < allChunks.length; i += 96) {
    const batch = allChunks.slice(i, i + 96);
    const embeddings = await embedWithCohere(
      batch.map(c => c.text),
      'search_document',
    );
    results.push(...batch.map((chunk, j) => ({ chunk, embedding: embeddings[j] })));
  }

  return results;
}

For AliveMCP's use case — embedding MCP server capability descriptions to find servers that perform similar functions — Titan V2 at 512 dimensions with normalize=true is the right choice. Server descriptions are short (50–200 tokens), well within Titan's limit, and the retrieval task is symmetric (queries and documents describe capabilities in similar language).

Vector storage: pgvector and OpenSearch k-NN

Embedding vectors must be stored somewhere searchable. Two common options for MCP servers on AWS are pgvector (PostgreSQL extension, runs on RDS or local SQLite-adjacent with pgvecto.rs) and OpenSearch Service with the k-NN plugin.

-- pgvector setup for Titan 1024-dim embeddings
CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE mcp_server_embeddings (
  id          TEXT PRIMARY KEY,
  slug        TEXT NOT NULL,
  description TEXT NOT NULL,
  embedding   vector(1024) NOT NULL,
  created_at  TIMESTAMP DEFAULT NOW()
);

-- HNSW index for approximate nearest neighbor search
-- ef_construction=64, m=16 are good defaults for 1024-dim
CREATE INDEX ON mcp_server_embeddings
  USING hnsw (embedding vector_cosine_ops)
  WITH (ef_construction = 64, m = 16);

-- Query: find 5 most similar servers to a query vector
SELECT slug, description,
       1 - (embedding <=> $1::vector) AS similarity
FROM   mcp_server_embeddings
ORDER BY embedding <=> $1::vector   -- <=> = cosine distance; lower = more similar
LIMIT  5;

// Node.js: inserting embeddings and querying via pgvector
import { Pool } from 'pg';

const pool = new Pool({ connectionString: process.env.DATABASE_URL });

async function upsertServerEmbedding(
  slug: string,
  description: string,
  embedding: number[],
): Promise {
  await pool.query(
    `INSERT INTO mcp_server_embeddings (id, slug, description, embedding)
     VALUES ($1, $2, $3, $4::vector)
     ON CONFLICT (id) DO UPDATE
     SET description = EXCLUDED.description,
         embedding   = EXCLUDED.embedding`,
    [slug, slug, description, JSON.stringify(embedding)],
  );
}

async function findSimilarServers(
  queryEmbedding: number[],
  limit = 5,
): Promise> {
  const { rows } = await pool.query(
    `SELECT slug,
            1 - (embedding <=> $1::vector) AS similarity
     FROM   mcp_server_embeddings
     ORDER BY embedding <=> $1::vector
     LIMIT  $2`,
    [JSON.stringify(queryEmbedding), limit],
  );
  return rows;
}

For MCP servers deployed on AWS Lambda without a persistent RDS connection, consider storing embeddings in S3 as a parquet file and loading them into memory on cold start — practical for corpora under 50k vectors at 1024 dimensions (about 200MB uncompressed). Larger corpora need OpenSearch Service with the k-NN plugin or Amazon Aurora PostgreSQL with pgvector.

Choosing between Titan V2 and Cohere Embed 3

For most MCP tools building their first semantic search feature, Titan V2 at 1024 dimensions with normalize=true is the simpler starting point — single text per call, no input_type to manage, direct cosine similarity from the dot product. Switch to Cohere when you need asymmetric search (retrieval quality is measurably better), multilingual support, or when storage cost at scale becomes a concern (binary embeddings reduce vector storage 32× at 5–10% quality cost).

Common errors and solutions