MCP Tools for Pinecone — Api-Key header auth, namespace-scoped vectors, batch upsert limits, metadata filtering, serverless vs pod indexes

Authentication — Api-Key header, not Authorization Bearer

Pinecone uses a proprietary Api-Key header — not the Authorization: Bearer pattern used by most APIs. Every request to both the control plane (api.pinecone.io) and the data plane (index-specific host) must include this header. The mismatch is the most common integration error: sending Authorization: Bearer {key} returns HTTP 401 with a generic "Unauthorized" message that gives no indication the header name itself is wrong.

import fetch from 'node-fetch';

const PINECONE_API_KEY = process.env.PINECONE_API_KEY;
const PINECONE_CONTROL_PLANE = 'https://api.pinecone.io';

// All Pinecone requests — both control-plane and data-plane — use Api-Key header
const pineconeHeaders = {
  'Api-Key': PINECONE_API_KEY,
  'Content-Type': 'application/json',
};

// Verify API key is set at startup
if (!PINECONE_API_KEY) {
  throw new Error('PINECONE_API_KEY environment variable is required');
}

There is no token expiry — API keys are long-lived and do not need to be refreshed. The key is tied to a Pinecone project (not an organization), so a single key can access all indexes within that project.

Index host resolution — data-plane calls go to the index host

Pinecone separates the control plane (api.pinecone.io — create/describe/delete indexes, list collections) from the data plane (upsert, query, fetch, delete vectors). Data-plane calls must go to the index-specific host URL, which takes the form {index-name}-{project-id}.svc.{environment}.pinecone.io for pod-based indexes, or a generated URL for serverless indexes. This host is returned by the describe-index endpoint.

// Resolve index host at MCP server startup — cache it, don't fetch on every call
async function resolveIndexHost(indexName: string): Promise<string> {
  const res = await fetch(`${PINECONE_CONTROL_PLANE}/indexes/${indexName}`, {
    headers: pineconeHeaders,
  });
  if (!res.ok) {
    const body = await res.text();
    throw new Error(`Failed to describe Pinecone index ${indexName}: HTTP ${res.status} — ${body}`);
  }
  const data = await res.json() as { host: string; status: { ready: boolean; state: string } };
  if (!data.status.ready) {
    throw new Error(`Pinecone index ${indexName} is not ready: state=${data.status.state}`);
  }
  return data.host; // e.g. "my-index-abc123.svc.us-east1-aws.pinecone.io"
}

// Cache at startup
const indexHost = await resolveIndexHost(process.env.PINECONE_INDEX_NAME!);

Calling data-plane endpoints against api.pinecone.io instead of the index host returns HTTP 404. This is one of the more confusing Pinecone errors because the control-plane API is working correctly — the fix is always to use the per-index host for vector operations.

Namespaces — vector partitioning with complete isolation

A namespace is a named partition within an index. Vectors upserted to namespace "tenant-abc" are completely invisible to queries against namespace "tenant-xyz", even if both are in the same index. Vector IDs are namespace-scoped — the same string ID can exist in multiple namespaces without collision. Namespaces are created automatically on first upsert and deleted automatically when their last vector is removed.

// Upsert with explicit namespace — omitting namespace puts vectors in the default (unnamed) namespace
async function upsertVectors(
  host: string,
  namespace: string,
  vectors: Array<{ id: string; values: number[]; metadata?: Record<string, unknown> }>
) {
  const res = await fetch(`https://${host}/vectors/upsert`, {
    method: 'POST',
    headers: pineconeHeaders,
    body: JSON.stringify({ vectors, namespace }),
  });
  if (!res.ok) throw new Error(`Upsert failed: ${res.status}`);
  return res.json(); // { upsertedCount: N }
}

// Query scoped to a specific namespace
async function queryVectors(
  host: string,
  namespace: string,
  queryVector: number[],
  topK: number,
  filter?: Record<string, unknown>
) {
  const res = await fetch(`https://${host}/query`, {
    method: 'POST',
    headers: pineconeHeaders,
    body: JSON.stringify({
      vector: queryVector,
      topK,
      namespace,
      filter,
      includeMetadata: true,
      includeValues: false, // omit raw vectors from results to reduce payload size
    }),
  });
  if (!res.ok) throw new Error(`Query failed: ${res.status}`);
  return res.json(); // { matches: [{ id, score, metadata }] }
}

Querying without a namespace field searches the default (unnamed) namespace only — it does not search all namespaces. To search all namespaces, you must query each namespace separately and merge results client-side. Use GET https://{host}/describe_index_stats to enumerate namespaces and their vector counts.

Batch upsert limits — 100 vectors or 2 MB per request

Pinecone enforces a maximum of 100 vectors per upsert request, with an additional 2 MB total payload limit. For bulk loads, implement a batching loop rather than a single large request. The 2 MB limit is typically hit first when vectors have large dimensions (e.g., 3072-dimensional OpenAI text-embedding-3-large vectors are 12 KB each as float32, so ~160 vectors hits 2 MB before the 100-vector count limit).

// Batch upsert with automatic chunking — handles both count and size limits
async function batchUpsert(
  host: string,
  namespace: string,
  vectors: Array<{ id: string; values: number[]; metadata?: Record<string, unknown> }>,
  batchSize = 100
) {
  let upsertedCount = 0;
  for (let i = 0; i < vectors.length; i += batchSize) {
    const batch = vectors.slice(i, i + batchSize);
    const result = await upsertVectors(host, namespace, batch);
    upsertedCount += result.upsertedCount;
    // Brief pause between batches to avoid rate limiting (serverless: 100 RPS default)
    if (i + batchSize < vectors.length) {
      await new Promise(r => setTimeout(r, 10));
    }
  }
  return upsertedCount;
}

Upsert is idempotent — sending the same vector ID again overwrites the previous values and metadata. This makes retry-safe bulk loads straightforward: checkpoint your progress by batch index and resume from the last successful batch on failure.

Metadata filtering — operators and indexed fields

Pinecone supports pre-filter metadata before vector similarity search, reducing the candidate pool before ANN lookup. Filters use MongoDB-style operators: $eq, $ne, $gt, $gte, $lt, $lte for scalar comparisons; $in and $nin for array membership; $and and $or for logical composition.

// Filter to vectors where category is "article" AND published_year >= 2024
// AND language is in ["en", "de"]
const filter = {
  $and: [
    { category: { $eq: 'article' } },
    { published_year: { $gte: 2024 } },
    { language: { $in: ['en', 'de'] } },
  ],
};

const queryBody = {
  vector: embeddingVector,
  topK: 10,
  namespace: 'documents',
  filter,
  includeMetadata: true,
};

// Important: metadata fields used in filters should be declared in the index's
// metadata_config.indexed array for pod-based indexes — unlisted fields are
// stored but not indexed, and filtering on them scans all vectors

On pod-based indexes, metadata fields must be declared in metadata_config.indexed at index creation time to be filterable without a full scan. On serverless indexes, all metadata is automatically indexed — no configuration needed, but metadata storage is billed. Filtering on a non-indexed metadata field in a pod index does not return an error; it silently performs a full scan, which can cause unexpected latency spikes at scale.

Serverless vs pod-based indexes

Serverless indexes scale automatically and charge per query (read units) and storage (write units). Pod-based indexes provision fixed capacity using pod types (s1, p1, p2) with configurable replicas — they provide predictable latency and throughput but require right-sizing. The key operational difference for MCP tools is that serverless indexes support only the cosine and dotproduct distance metrics, while pod indexes also support euclidean. Serverless indexes also have a default rate limit of 100 requests per second per project; pod indexes are rate-limited only by pod capacity.

// Create a serverless index (current recommended default for new projects)
const createServerless = await fetch(`${PINECONE_CONTROL_PLANE}/indexes`, {
  method: 'POST',
  headers: pineconeHeaders,
  body: JSON.stringify({
    name: 'my-index',
    dimension: 1536,            // must match your embedding model output dimension
    metric: 'cosine',           // cosine | dotproduct (serverless); also euclidean for pod
    spec: {
      serverless: {
        cloud: 'aws',           // aws | gcp | azure
        region: 'us-east-1',
      },
    },
  }),
});

// Create a pod-based index (for predictable latency at high QPS)
const createPod = await fetch(`${PINECONE_CONTROL_PLANE}/indexes`, {
  method: 'POST',
  headers: pineconeHeaders,
  body: JSON.stringify({
    name: 'my-pod-index',
    dimension: 1536,
    metric: 'cosine',
    spec: {
      pod: {
        environment: 'us-east1-gcp',
        pod_type: 'p1.x1',     // p1.x1 | p1.x2 | p2.x1 | s1.x1 etc.
        pods: 1,
        replicas: 1,
        shards: 1,
      },
    },
  }),
});

Index creation is asynchronous — the API returns HTTP 201 immediately but the index takes 1–5 minutes to become ready. Poll GET /indexes/{name} and wait for status.state === "Ready" before attempting data-plane operations. Attempting to upsert into an initializing index returns HTTP 503.

Health probe — describe_index and describe_index_stats

Pinecone has two useful health endpoints. GET https://api.pinecone.io/indexes/{name} checks index existence and readiness — it validates credentials AND confirms the index is operational. POST https://{host}/describe_index_stats provides deeper operational health: total vector count, namespace distribution, and index fullness (ratio of current vectors to pod capacity, relevant for pod indexes approaching their limit).

async function checkPineconeHealth(indexName: string, host: string): Promise<{
  ready: boolean;
  totalVectorCount: number;
  namespaces: Record<string, { vectorCount: number }>;
  indexFullness: number;
}> {
  // Step 1: control-plane check (credentials + index existence + state)
  const describeRes = await fetch(
    `${PINECONE_CONTROL_PLANE}/indexes/${indexName}`,
    { headers: pineconeHeaders }
  );
  if (describeRes.status === 404) {
    throw new Error(`Pinecone index "${indexName}" does not exist`);
  }
  if (describeRes.status === 401) {
    throw new Error('Pinecone API key is invalid or missing');
  }
  const indexData = await describeRes.json();
  if (!indexData.status?.ready) {
    return { ready: false, totalVectorCount: 0, namespaces: {}, indexFullness: 0 };
  }

  // Step 2: data-plane stats (vector counts, namespace list, fullness)
  const statsRes = await fetch(`https://${host}/describe_index_stats`, {
    method: 'POST',
    headers: pineconeHeaders,
    body: JSON.stringify({}),
  });
  const stats = await statsRes.json();
  return {
    ready: true,
    totalVectorCount: stats.totalVectorCount,
    namespaces: stats.namespaces ?? {},
    indexFullness: stats.indexFullness ?? 0, // 0.0 to 1.0; >0.8 means approaching pod capacity
  };
}

An indexFullness above 0.8 on a pod index means queries will start returning fewer than topK results and latency increases as the HNSW graph degrades under load. For serverless indexes, indexFullness is always 0 — capacity is unlimited. Monitor this metric for pod indexes and provision additional replicas or upgrade pod type before hitting 1.0.

Common integration errors

• Wrong auth header name — Authorization: Bearer {key} returns HTTP 401. Must be Api-Key: {key}.
• Data-plane calls to control-plane host — POST https://api.pinecone.io/vectors/upsert returns HTTP 404. Use the per-index host from GET /indexes/{name}.
• Querying without namespace, expecting all vectors — queries without namespace search only the default namespace. Specify namespace explicitly or enumerate all namespaces from describe_index_stats.
• Upsert batch over 100 vectors — returns HTTP 400 "Request exceeds maximum number of vectors per upsert." Split into batches of ≤100.
• Dimension mismatch — upserting a vector with dimension ≠ index dimension returns HTTP 400 "Vector dimension does not match the dimension of the index." Always assert dimension match at startup by comparing embedding model output against indexData.dimension.
• Querying index in Initializing state — returns HTTP 503. Wait for status.state === "Ready" before any data-plane calls.