MCP server Ollama integration

Setup: no SDK required

Ollama exposes a REST API on localhost:11434. You don't need an npm package — the built-in fetch API (available in Node.js 18+) is sufficient:

import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
import { z } from "zod";

const OLLAMA_BASE = process.env.OLLAMA_HOST ?? "http://localhost:11434";

const server = new McpServer({ name: "ollama-tools", version: "1.0.0" });

// Shared fetch helper with timeout
async function ollamaFetch(path: string, body?: object): Promise<Response> {
  const controller = new AbortController();
  const timeout = setTimeout(() => controller.abort(), 120_000); // 2 min timeout

  try {
    const res = await fetch(`${OLLAMA_BASE}${path}`, {
      method: body ? "POST" : "GET",
      headers: body ? { "Content-Type": "application/json" } : undefined,
      body: body ? JSON.stringify(body) : undefined,
      signal: controller.signal,
    });
    if (!res.ok) {
      const text = await res.text();
      throw new Error(`Ollama ${path} returned ${res.status}: ${text}`);
    }
    return res;
  } finally {
    clearTimeout(timeout);
  }
}

The OLLAMA_HOST environment variable lets you point MCP tools at a remote Ollama instance (on a GPU box, for example) without code changes. The default localhost:11434 covers local dev.

Health check tool

Ollama is a separate process — it can crash, fail to start, or be unavailable while your MCP server is running. Always check health before inference rather than letting inference calls time out after 2 minutes:

server.tool(
  "check_ollama_health",
  "Verify that the Ollama server is running and return the list of available models",
  {},
  async () => {
    try {
      const res = await ollamaFetch("/api/tags");
      const data = await res.json() as { models: Array<{ name: string; size: number; modified_at: string }> };

      return {
        content: [{
          type: "text",
          text: JSON.stringify({
            status: "running",
            host: OLLAMA_BASE,
            models: data.models.map(m => ({
              name: m.name,
              size_gb: (m.size / 1e9).toFixed(1),
              modified_at: m.modified_at,
            })),
          }),
        }],
      };
    } catch (err) {
      return {
        isError: true,
        content: [{
          type: "text",
          text: JSON.stringify({
            status: "unreachable",
            host: OLLAMA_BASE,
            error: err instanceof Error ? err.message : String(err),
            hint: "Run `ollama serve` to start the Ollama server",
          }),
        }],
      };
    }
  }
);

Call check_ollama_health at MCP server startup and log the result. For production deployments, add this as a liveness check in your process monitor so restarts are automatic when Ollama goes down.

Chat endpoint tool

The /api/chat endpoint takes a messages array (same shape as OpenAI chat format) and returns the response. The critical MCP-specific flag: "stream": false. Without it, Ollama returns a stream of newline-delimited JSON objects — your await res.json() will parse only the first object and miss the rest.

server.tool(
  "chat_with_model",
  "Send a conversation to an Ollama model and return the response",
  {
    model: z.string().describe("Ollama model name — e.g. llama3.2, mistral, gemma3, qwen2.5"),
    messages: z.array(z.object({
      role: z.enum(["system", "user", "assistant"]),
      content: z.string(),
    })).min(1).describe("Conversation history — the last message should be role: user"),
    temperature: z.number().min(0).max(2).default(0.7).optional(),
    num_ctx: z.number().int().optional()
      .describe("Override context window size — defaults to model's built-in limit"),
  },
  async ({ model, messages, temperature, num_ctx }) => {
    const options: Record<string, unknown> = {};
    if (temperature !== undefined) options.temperature = temperature;
    if (num_ctx !== undefined) options.num_ctx = num_ctx;

    try {
      const res = await ollamaFetch("/api/chat", {
        model,
        messages,
        stream: false,  // CRITICAL — must be false for MCP tools
        options: Object.keys(options).length > 0 ? options : undefined,
      });

      const data = await res.json() as {
        message: { role: string; content: string };
        done: boolean;
        total_duration: number;
        prompt_eval_count: number;
        eval_count: number;
      };

      return {
        content: [{
          type: "text",
          text: JSON.stringify({
            reply: data.message.content,
            model,
            prompt_tokens: data.prompt_eval_count,
            output_tokens: data.eval_count,
            total_duration_ms: Math.round(data.total_duration / 1e6),
          }),
        }],
      };
    } catch (err) {
      return {
        isError: true,
        content: [{ type: "text", text: `Ollama chat error: ${err instanceof Error ? err.message : String(err)}` }],
      };
    }
  }
);

Generate endpoint tool

The /api/generate endpoint takes a single text prompt (not a messages array) and is useful for completion-style tasks — filling in code, generating raw text without a conversational structure:

server.tool(
  "generate_with_model",
  "Complete a text prompt using an Ollama model — useful for code generation, summarization, extraction",
  {
    model: z.string().describe("Ollama model name"),
    prompt: z.string().describe("Text prompt to complete"),
    system: z.string().optional().describe("System context to prepend to the prompt"),
    max_tokens: z.number().int().min(1).max(8192).default(512)
      .describe("Maximum number of tokens to generate (num_predict)"),
  },
  async ({ model, prompt, system, max_tokens }) => {
    try {
      const res = await ollamaFetch("/api/generate", {
        model,
        prompt,
        system,
        stream: false,
        options: { num_predict: max_tokens },
      });

      const data = await res.json() as {
        response: string;
        done: boolean;
        done_reason: string;
        prompt_eval_count: number;
        eval_count: number;
      };

      return {
        content: [{
          type: "text",
          text: JSON.stringify({
            text: data.response,
            done_reason: data.done_reason,
            prompt_tokens: data.prompt_eval_count,
            output_tokens: data.eval_count,
          }),
        }],
      };
    } catch (err) {
      return {
        isError: true,
        content: [{ type: "text", text: `Ollama generate error: ${err instanceof Error ? err.message : String(err)}` }],
      };
    }
  }
);

The done_reason field tells you why generation stopped: "stop" means the model finished naturally, "length" means num_predict was hit (increase max_tokens if you're seeing truncated output).

Model management tools

MCP tools for pulling, listing, and removing models turn Ollama model management into agent-accessible operations:

server.tool(
  "pull_model",
  "Download an Ollama model from the registry (ollama.com/library). Blocks until complete.",
  {
    model: z.string().describe("Model name to pull — e.g. 'llama3.2', 'mistral:7b', 'qwen2.5:14b'"),
  },
  async ({ model }) => {
    try {
      // pull streams progress — we collect the final status line
      const res = await ollamaFetch("/api/pull", { name: model, stream: false });
      const data = await res.json() as { status: string };
      return {
        content: [{ type: "text", text: JSON.stringify({ model, status: data.status }) }],
      };
    } catch (err) {
      return {
        isError: true,
        content: [{ type: "text", text: `Pull failed: ${err instanceof Error ? err.message : String(err)}` }],
      };
    }
  }
);

server.tool(
  "delete_model",
  "Remove an Ollama model from local storage to reclaim disk space",
  {
    model: z.string().describe("Model name to delete — must match exactly as returned by check_ollama_health"),
  },
  async ({ model }) => {
    try {
      const res = await fetch(`${OLLAMA_BASE}/api/delete`, {
        method: "DELETE",
        headers: { "Content-Type": "application/json" },
        body: JSON.stringify({ name: model }),
      });
      if (res.status === 404) {
        return {
          isError: true,
          content: [{ type: "text", text: `Model '${model}' not found — run check_ollama_health to see available models` }],
        };
      }
      return {
        content: [{ type: "text", text: `Deleted model: ${model}` }],
      };
    } catch (err) {
      return {
        isError: true,
        content: [{ type: "text", text: `Delete failed: ${err instanceof Error ? err.message : String(err)}` }],
      };
    }
  }
);

Context window limits by model

Unlike OpenAI and Anthropic where all current models have 128k–200k token windows, Ollama models vary dramatically. Sending a prompt that exceeds the model's context window causes silent truncation — the model silently ignores the oldest tokens rather than returning an error.

You can override num_ctx in the options object, but going above the model's maximum has no effect. Going above available GPU/CPU memory causes OOM errors. Check prompt_eval_count in responses — if it equals num_ctx, the prompt was silently truncated.

When to use Ollama vs cloud APIs in an MCP server

The practical rule: use Ollama in local development (zero cost, no key management) and cloud APIs in production (reliability, quality). For privacy-sensitive production MCP servers (medical, legal, financial data), Ollama on a private GPU box is the correct architecture — no data leaves the network perimeter.

Frequently asked questions

Further reading

• MCP tools for the OpenAI API — chat completions, function calling, rate limits
• MCP tools for the Anthropic/Claude API — Messages API, tool use, prompt caching
• MCP tools for AWS Bedrock — Converse API, IAM auth, cross-region inference
• Ollama as an MCP server — using the registry-listed Ollama MCP integration
• Streaming responses in MCP servers
• Token budget management for MCP servers
• MCP server health checks — transport, protocol, and dependency probing
• AliveMCP — production protocol monitoring for MCP servers