Guide · Data Serialization Integration

MCP Server MessagePack — compact binary encoding, ext types, Redis binary values

Three MessagePack behaviours trip up MCP server authors: integers near type boundaries decode as different JavaScript types depending on their value — MessagePack encodes integers in the smallest format that fits (positive fixint, uint8, uint16, uint32, uint64), and 255 encodes as uint8 while 256 encodes as uint16; both decode to JavaScript number, but values > 2³² encode as uint64 which @msgpack/msgpack decodes as bigint by default, and JSON.stringify cannot serialise bigint — always check for typeof value === 'bigint' before returning from tool handlers; MessagePack bin type decodes to Uint8Array, not a string — Redis RESP3 returns binary hash values as Buffer objects which MessagePack stores as bin; if you pass a Uint8Array through JSON.stringify it becomes an object {"0":72,"1":101,...} not a base64 string; and the msgpackr library has a different default behaviour than @msgpack/msgpack for struct packing — msgpackr with useRecords: true uses a record extension that is incompatible with any other MessagePack implementation.

TL;DR

Use @msgpack/msgpack for cross-language MessagePack compatibility. Always convert bigint to string before passing decoded values to JSON.stringify. Convert Uint8Array/Buffer fields to base64 strings with Buffer.from(value).toString('base64'). Use custom ext types for Date serialisation if you need millisecond-precision timestamps. Avoid msgpackr record mode when interoperating with Python/Rust/Go consumers.

When to use MessagePack instead of JSON in MCP tool responses

JSON is the right default for MCP tool responses: it is human-readable in logs, natively supported by every agent runtime, and fast enough for payloads under a few kilobytes. MessagePack becomes useful in MCP servers that act as caches or proxies for large data: telemetry rows, feature vectors, bulk export results, or responses from backends that already speak MessagePack (NATS JetStream objects, Redis values encoded by a Python publisher). In those cases, MessagePack can cut serialisation size by 30–60% compared to JSON for numeric-heavy payloads and by 10–20% for string-heavy payloads.

import { encode, decode, ExtensionCodec } from '@msgpack/msgpack';

// Custom ext type for JavaScript Date — ext type 1 encodes milliseconds as int64
const DATE_EXT_TYPE = 1;
const extensionCodec = new ExtensionCodec();

extensionCodec.register({
  type: DATE_EXT_TYPE,
  encode: (value: unknown): Uint8Array | null => {
    if (value instanceof Date) {
      const buffer = new ArrayBuffer(8);
      new DataView(buffer).setBigInt64(0, BigInt(value.getTime()));
      return new Uint8Array(buffer);
    }
    return null; // not a Date — skip this encoder
  },
  decode: (data: Uint8Array): Date => {
    const ms = new DataView(data.buffer, data.byteOffset, data.byteLength).getBigInt64(0);
    return new Date(Number(ms));
  },
});

// Safe encode for MCP tool responses — ensures output is always JSON-serialisable
function encodeForMcp(value: unknown): string {
  // For storage/cache: encode to binary
  const binary = encode(value, { extensionCodec });
  // Return base64-encoded binary as a string (MCP tool responses are text)
  return Buffer.from(binary).toString('base64');
}

function decodeFromCache(base64: string): unknown {
  const binary = Buffer.from(base64, 'base64');
  return decode(binary, { extensionCodec });
}

// Safe serialiser for tool response content — handles bigint and Uint8Array
function toJsonSafe(value: unknown): string {
  return JSON.stringify(value, (_key, val) => {
    if (typeof val === 'bigint') return val.toString();
    if (val instanceof Uint8Array || Buffer.isBuffer(val)) {
      return Buffer.from(val).toString('base64');
    }
    return val;
  });
}

// MCP tool: fetch and decode a MessagePack value from Redis
import { createClient } from 'redis';

const redis = createClient({ url: process.env.REDIS_URL });
await redis.connect();

server.tool(
  'get_cached_report',
  { report_id: z.string() },
  async ({ report_id }) => {
    // Redis returns Buffer when the value is binary
    const raw = await redis.getBuffer(`report:${report_id}`);
    if (!raw) {
      return { content: [{ type: 'text', text: JSON.stringify({ error: 'not_found' }) }] };
    }

    // Decode MessagePack binary
    const decoded = decode(raw, { extensionCodec });

    // Use safe serialiser — decoded may contain bigint or Uint8Array fields
    return {
      content: [{ type: 'text', text: toJsonSafe(decoded) }],
    };
  }
);

Encoding large tool responses for inter-service caching

MCP servers that aggregate data from multiple downstream services often cache intermediate results to speed up repeated agent queries. MessagePack is a good choice for this cache layer: it is faster to encode/decode than JSON for numeric-heavy payloads and produces smaller values that stay under Redis memory limits. The key discipline is to always decode the MessagePack to a plain JS object before constructing the MCP tool response, never to return binary directly.

interface ReportRow {
  user_id: string;
  event_count: number;
  revenue_cents: number;
  sessions: number[];
  created_at: Date;  // will be encoded with our custom Date ext type
}

// Store a computed report in Redis as MessagePack
async function cacheReport(reportId: string, rows: ReportRow[], ttlSeconds = 3600): Promise<void> {
  const encoded = encode({ rows, cached_at: new Date() }, { extensionCodec });
  await redis.set(`report:${reportId}`, Buffer.from(encoded), { EX: ttlSeconds });
}

// MCP tool: generate or retrieve cached analytics report
server.tool(
  'get_analytics_report',
  { report_id: z.string(), force_refresh: z.boolean().default(false) },
  async ({ report_id, force_refresh }) => {
    if (!force_refresh) {
      const cached = await redis.getBuffer(`report:${report_id}`);
      if (cached) {
        const { rows, cached_at } = decode(cached, { extensionCodec }) as { rows: ReportRow[]; cached_at: Date };
        return {
          content: [{
            type: 'text',
            text: toJsonSafe({
              source: 'cache',
              cached_at: cached_at.toISOString(),
              row_count: rows.length,
              rows,
            }),
          }],
        };
      }
    }

    // Cache miss — recompute (expensive)
    const rows = await computeAnalyticsReport(report_id);
    await cacheReport(report_id, rows);

    return {
      content: [{
        type: 'text',
        text: toJsonSafe({ source: 'computed', row_count: rows.length, rows }),
      }],
    };
  }
);

// Size comparison helper — useful for deciding when MessagePack is worth it
function compareSize(value: unknown): { json_bytes: number; msgpack_bytes: number; savings_pct: number } {
  const json_bytes   = Buffer.byteLength(JSON.stringify(value));
  const msgpack_bytes = encode(value).byteLength;
  const savings_pct  = Math.round((1 - msgpack_bytes / json_bytes) * 100);
  return { json_bytes, msgpack_bytes, savings_pct };
}
// Typical results:
// Numeric array of 1000 ints: JSON ~5KB, MessagePack ~3KB → 40% smaller
// String-heavy user objects:  JSON ~8KB, MessagePack ~7KB → 12% smaller
// Mixed report rows:          JSON ~20KB, MessagePack ~14KB → 30% smaller

Cross-language compatibility — interoperating with Python and Go publishers

MCP servers in Node.js often consume MessagePack-encoded data written by Python or Go services. The core MessagePack specification (types: nil, bool, int, float, str, bin, array, map, ext) is compatible across all mainstream libraries. The pitfalls are in the extensions: msgpackr's record mode (ext type 0) is Node.js-only; Python's msgpack library uses ext type 5 for its Timestamp; and Go's vmihailenco/msgpack uses different struct tag conventions. Stick to the core types only when interoperating — use raw map + str for objects, int/float for numbers, and agree on a shared ext type ID for any custom types.

// Python publisher side (for reference):
// import msgpack
// data = {'user_id': 'u123', 'score': 99.5, 'tags': ['a', 'b']}
// packed = msgpack.packb(data, use_bin_type=True)  # use_bin_type=True: str → str, bytes → bin
//
// Node.js consumer — @msgpack/msgpack is compatible with Python msgpack default settings
import { decode } from '@msgpack/msgpack';

async function decodePythonPublisherMessage(buffer: Buffer): Promise<Record<string, unknown>> {
  // @msgpack/msgpack decodes str → string, bin → Uint8Array, map → Map or object
  const decoded = decode(buffer, {
    // forceIntegerToFloat: false — keep integer types as-is (default)
    // rawStrings: false — decode str as JS string (default, matches Python use_bin_type=True)
  });

  if (typeof decoded !== 'object' || decoded === null || Array.isArray(decoded)) {
    throw new Error(`Expected MessagePack map, got ${typeof decoded}`);
  }

  // Convert any Uint8Array fields (Python bytes) to base64 for JSON safety
  const result: Record<string, unknown> = {};
  for (const [key, value] of Object.entries(decoded as Record<string, unknown>)) {
    if (value instanceof Uint8Array) {
      result[key] = Buffer.from(value).toString('base64');
    } else if (typeof value === 'bigint') {
      result[key] = value.toString();
    } else {
      result[key] = value;
    }
  }
  return result;
}

// NATS JetStream with MessagePack-encoded payloads
import { connect as natsConnect, StringCodec } from 'nats';

async function subscribeNatsMsgpack(subject: string, handler: (data: unknown) => void): Promise<void> {
  const nc = await natsConnect({ servers: process.env.NATS_SERVERS!.split(',') });
  const js = nc.jetstream();

  const consumer = await js.consumers.get('EVENTS', subject);
  const messages = await consumer.consume({ max_messages: 100 });

  for await (const msg of messages) {
    try {
      const decoded = decode(msg.data, { extensionCodec });
      handler(decoded);
      msg.ack();
    } catch (err) {
      console.error('MessagePack decode failed:', err);
      msg.nak();
    }
  }
}