Feature Flags & Experimentation · 2026-07-14 · Feature Flags arc
MCP Tools for Feature Flag Systems: Dual Credential Architecture, Evaluation Health vs Process Health, and the Silent Impression Gap Across LaunchDarkly, Unleash, Flagsmith, OpenFeature, and Split.io
Every feature flag platform has two separate auth surfaces that require different credentials and serve different purposes — and swapping them produces auth errors that look like permission failures rather than credential format errors. LaunchDarkly requires sdk-xxx keys for the streaming connection and Authorization: Bearer <access-token> for REST API v2 management operations — passing the SDK key to the REST API returns 401. Unleash requires Authorization: Bearer <admin-token> for the Admin API but Authorization: <client-token> with no Bearer prefix for the Client API — adding the prefix to a Client API call returns 401 on some versions. Flagsmith uses X-Environment-Key: ser.xxx for environment-scoped SDK reads and Authorization: Api-Key xxx for management operations — each is scoped to a different level of the hierarchy. OpenFeature abstracts away provider credentials behind the provider interface, but the credential confusion moves one level up: setProvider() is fire-and-forget and evaluations during initialization return default values silently, while setProviderAndWait() blocks until the provider is READY. Split.io separates SDK API keys (for streaming synchronization of split definitions) from Admin API keys (for REST API v2 management) — using a browser-side SDK key in an MCP server produces reduced-permission behavior with no obvious error. Beyond credential architecture, all five platforms share a second structural pattern: their built-in health endpoints report process liveness only, not evaluation health — the flag evaluation path can be silently broken while /healthcheck, /health, and getProviderStatus() all return healthy. And a third pattern hides deeper: impression and event delivery — the mechanism that powers A/B experiment measurement — can fail silently while evaluations continue returning correct treatments, corrupting experiment data without any observable signal at the evaluation layer. This synthesis covers all three patterns across all five platforms, with probe designs that catch what built-in health endpoints miss.
TL;DR
Five feature flag platforms, three shared structural patterns. (1) Dual credential architecture — SDK keys and management API keys cannot be swapped: LaunchDarkly SDK keys (sdk-xxx) authenticate to the streaming connection only; access tokens authenticate to REST API v2 (/api/v2/) — 401 when swapped; Unleash Admin API requires Bearer <admin-token>; Client API requires the raw client token with no Bearer prefix — the same 401 for wrong format hides a header format bug rather than a bad credential; Flagsmith environment keys (ser.xxx in X-Environment-Key) scope to one environment; management API keys (Api-Key xxx) scope to org/project — neither works for the other's endpoints; OpenFeature abstracts credentials into provider packages — the credential bug becomes "using setProvider() instead of setProviderAndWait()" which produces silent defaults rather than 401s; Split.io SDK keys authenticate to the synchronization stream; Admin API keys authenticate to api.split.io/internal/api/v2/ — a server-side SDK key used in a browser context silently reduces permissions. (2) Process health endpoints do not verify evaluation health: LaunchDarkly's GET /healthcheck returns healthy while the SDK streaming connection reconnects every 30 seconds and flag updates take 5× longer to propagate; Unleash's GET /health → {"health":"GOOD"} returns healthy while the Client API's database reads (what SDKs actually poll) fail because a read replica is down; Flagsmith's GET /health/ → {"status":"ok"} returns healthy while the segment matching engine returns incorrect identity evaluations after a bad migration; OpenFeature's ProviderStatus.READY can coexist with evaluations returning defaults because targetingKey is missing from context and providers silently fall back to errorCode: TARGETING_KEY_MISSING; Split.io's SDK_READY event fires while the streaming connection for real-time split updates is broken and the SDK silently falls back to 60-second polling. Composite probes that exercise the evaluation path directly are required for all five platforms. (3) Impression delivery can fail while evaluations succeed: LaunchDarkly's event flush buffer can fail silently — client.flush() before process exit is required to avoid losing buffered experiment events; Unleash's impression metrics (GET /api/admin/metrics/feature-toggles/raw) can show zero counts even when evaluations are succeeding because the metrics aggregation pipeline has stopped; Flagsmith's POST /identities/ is a write operation that creates and persists identities — using it as a health probe adds unnecessary write load; Split.io's track() returns false (not throw) when the SDK is not ready, silently dropping all experiment events without any error signal; OpenFeature's error hook fires on evaluation errors but impression delivery is fully provider-specific — no standard way to verify events are flowing. Monitor impression delivery independently from flag evaluation: broken impression delivery corrupts experiment data while every evaluation API call returns HTTP 200. Probe all three layers with AliveMCP.
Pattern 1: The Dual Credential Architecture — SDK Keys and Management API Keys Cannot Be Swapped
Every feature flag platform separates its evaluation surface from its management surface, and each surface requires a different type of credential. This is not arbitrary — the SDK streaming connection needs a credential scoped to a specific environment and project, optimized for low-latency streaming; the management API needs a credential with write permissions across the flag catalog, secured for administrative operations. The two credential types have different scopes, different permission models, and different auth mechanisms — and swapping them produces auth errors that look like permission failures rather than credential format errors.
The diagnostic challenge is that a developer who uses a LaunchDarkly SDK key (sdk-xxx) to call the REST API v2 gets back HTTP 401 — which looks identical to an expired access token or an insufficient-permission access token. There's no error body field that says "you used an SDK key on a management endpoint." The same ambiguity applies across every platform: the error shape for "wrong credential type" and "correct credential type, wrong permission level" is identical. MCP tools must document which credential type they require and validate the credential format before attempting API calls.
LaunchDarkly has the most explicit two-surface model. The SDK key (sdk-xxx) is an environment-scoped credential used exclusively with the server-side SDK to receive flag configurations over a persistent SSE streaming connection. It has no REST API access. The access token (personal or service account) is used for REST API v2 calls at https://app.launchdarkly.com/api/v2/ — it can have reader, writer, or admin role and is not scoped to a single environment unless policies restrict it. The credential type can be verified before any flag operation: GET /api/v2/caller-identity returns the token's type, role, and project access. An SDK key passed to this endpoint returns 401 with no guidance on the credential type mismatch.
The SDK key's sole purpose is the streaming connection. Each LaunchDarkly.init(sdkKey) call opens a new SSE connection. Running init() per request instead of using a module-level singleton multiplies streaming connections and duplicates event delivery. The module-level singleton must call await ldClient.waitForInitialization({ timeout: 10 }) before serving requests — without it, the flag store may be empty and every variation() call silently returns the default value. waitForInitialization() is the initialization barrier; skipping it turns "flag not yet loaded" into an invisible bug where all users get the default variation regardless of targeting rules.
Unleash has the most confusing dual-credential pattern because the credential format difference is subtle: the Admin API requires Authorization: Bearer <admin-token> while the Client API requires Authorization: <client-token> — the identical string, minus the "Bearer " prefix. Adding "Bearer " to a Client API call returns 401 on some Unleash versions, making the error look like an invalid client token rather than a header format error. The reason for the distinction is architectural: the Client API is designed for SDK polling compatibility, and some Unleash SDK versions predate the Bearer convention. Stripping the prefix maintains backward compatibility, but the error surface for getting it wrong is opaque.
Unleash also has an important implication for the Admin API beyond simple flag management. The Admin API's /api/admin/ui-config endpoint requires a valid admin token and reads configuration from the database. If this call succeeds, it confirms both token validity and database read connectivity — it's a more meaningful health signal than GET /health alone. Similarly, /api/admin/instance-admin/statistics confirms database connectivity via a different read path (toggle counts, user counts, project counts). Two separate database-touching Admin API calls make a stronger composite health probe than either alone.
Flagsmith has a three-level credential hierarchy: the Environment key (ser.xxx in the X-Environment-Key header) scopes to a single environment within a project; the management API key (Authorization: Api-Key xxx) scopes to org/project level and enables cross-environment operations, segment management, and audit log access. Neither works for the other's endpoints — using an environment key to call management endpoints returns 403, using a management key in the X-Environment-Key header may return flags from an unexpected scope. Flagsmith also has a client-side key type (pk.xxx, for public/browser use) distinct from the server-side key (ser.xxx) — using a client-side key in an MCP server silently reduces security guarantees (client-side keys are designed to be exposed publicly and have reduced access).
The scope implication of Flagsmith's environment key has a quiet failure mode: if a flag was added to the project after the environment key was scoped, GET /flags/ returns the flag in its default state (disabled, null value) with no indication that a newer environment or set of flags exists. There is no "you're seeing a partial view" signal — the response looks complete. MCP tools that need cross-environment visibility must use the management key.
OpenFeature abstracts away provider-specific credentials — the credential for the underlying flag system (LaunchDarkly SDK key, Unleash admin token, Flagsmith environment key) is passed to the provider package, not to OpenFeature directly. But OpenFeature introduces its own form of credential-adjacent initialization bug: OpenFeature.setProvider(provider) is asynchronous and fire-and-forget — the provider initializes in the background. Evaluations called before the provider reaches READY return the default value with reason PROVIDER_NOT_READY, not an error. The default value is indistinguishable from a correctly evaluated value if the caller doesn't inspect the reason.
await OpenFeature.setProviderAndWait(provider) is the correct initialization pattern — it blocks until provider.initialize() resolves and the provider transitions to READY. In an MCP server that starts handling tool calls immediately on startup, using setProvider() instead of setProviderAndWait() creates a window — potentially seconds long — where every flag evaluation silently returns defaults. The provider's initialization is usually network-bound (connecting to LaunchDarkly's streaming endpoint, polling Unleash's Client API), so this window can be several seconds on cold start.
Split.io separates the server-side SDK key (used by the Node.js SDK to synchronize split definitions from Split.io's CDN and streaming API) from the Admin API key (used for management operations at api.split.io/internal/api/v2/). One critical distinction that has no parallel in other platforms: Split.io also has browser-side SDK keys, which are distinct from server-side keys. A browser-side key used in an MCP server (server-side code) silently produces reduced-permission behavior — certain split evaluation features are unavailable, and impression delivery works differently. The key type is determined at creation time in the Split.io console and is not detectable from the key format.
Split.io's initialization must also be treated as a blocking operation: the SDK fires SDK_READY when it has synchronized all split definitions and segment memberships from Split.io's servers, and SDK_READY_TIMED_OUT (at 1.5 seconds by default) if synchronization doesn't complete in time. Evaluations before SDK_READY return "control" — not the configured treatment, not an error that throws, but the silent error sentinel. Setting startup.readyTimeout to 5 seconds reduces false SDK_READY_TIMED_OUT fires on slow network starts without indefinitely blocking initialization.
| Platform | SDK/Evaluation Credential | Management API Credential | Swap Error Shape | Initialization Barrier |
|---|---|---|---|---|
| LaunchDarkly | sdk-xxx (streaming only) |
Bearer <access-token> → REST API v2 |
HTTP 401, no format hint | await ldClient.waitForInitialization() |
| Unleash | Authorization: <client-token> (no Bearer) |
Authorization: Bearer <admin-token> |
HTTP 401, looks like bad token | SDK: await unleash.start() |
| Flagsmith | X-Environment-Key: ser.xxx |
Authorization: Api-Key xxx |
HTTP 403 or wrong-scope response | SDK cache refresh interval |
| OpenFeature | Provider-specific (passed to provider) | Provider-specific (management API) | Silent defaults (not 401) | await setProviderAndWait(provider) |
| Split.io | SDK API key (streaming) | Admin API key → api.split.io/internal/api/v2/ |
HTTP 401, reduced permissions | SDK_READY event (reject on SDK_READY_TIMED_OUT) |
Pattern 2: Process Health Endpoints Do Not Verify Evaluation Health
Every feature flag platform has a health endpoint. Every platform's health endpoint reports process liveness — whether the server process is running and reachable. No platform's built-in health endpoint verifies the full evaluation path: whether the flag store has been populated, whether the database is serving flag configurations to SDKs, whether segment evaluation rules are computing correctly, whether the streaming connection is delivering real-time flag updates, or whether the evaluation API is returning non-default values. The health endpoint and the evaluation path fail independently.
This is the most operationally dangerous pattern in feature flag systems: an Unleash instance whose database read replica has failed returns {"health":"GOOD"} because the process is running and its write path (used for admin operations) remains healthy. But every SDK that polls the Client API for flag configurations is now reading from a stale cache — they will keep evaluating based on the configurations they last synchronized, with no indication that they're working from hours-old data. A LaunchDarkly SDK whose streaming connection is reconnecting every 30 seconds (indicating backend pressure or network instability) sees healthy responses from every management API call, because the management API is independent of the streaming endpoint.
LaunchDarkly's evaluation health has three independent components that GET /healthcheck cannot see. First, the streaming connection status: the Node.js SDK can report isInitialized() === true after initially connecting while the connection has subsequently been disconnecting and reconnecting frequently. Second, the flag store population: isInitialized() === true can coexist with a flag store that has zero entries — this happens when the SDK key belongs to an environment with no flags, or when the SDK connected to a streaming endpoint that sent an empty snapshot. Third, the REST API token scope: an access token can have reader role limited to a specific project, which makes it appear valid for GET /api/v2/caller-identity while returning 403 on flag operations in other projects.
The composite probe for LaunchDarkly requires three checks in parallel: (1) GET /api/v2/caller-identity to verify REST API token validity and scope, (2) ldClient.isInitialized() to verify the streaming connection was established, and (3) client.allFlagsState({kind:'user', key:'__health__'}).toJSON() with a count of non-metadata keys to verify the flag store has content. A flag store count that drops significantly compared to a baseline is an early signal of bulk flag archival or environment misconfiguration — catchable before users notice anything.
Unleash's health surface is the starkest example of the process-health-vs-evaluation-health split. A self-hosted Unleash deployment running PostgreSQL with a primary write node and a read replica: GET /health → {"health":"GOOD"} reflects the primary node's liveness. But Unleash's Client API (/api/client/features) — the endpoint that SDKs poll for toggle configurations — reads from the database. If the read replica fails, the Client API returns errors while the primary's health check returns green. Every running SDK in your infrastructure silently falls back to its last cached configuration, serving evaluations based on potentially hours-old toggle definitions with no error in your monitoring dashboard.
The comprehensive Unleash health probe requires three independent checks: (1) GET /api/admin/ui-config with the admin token — exercises the admin auth path and reads configuration from the primary database; (2) GET /api/client/features with the client token — exercises the exact path that SDKs use for toggle configuration synchronization, including the read replica if one is in use; (3) GET /api/admin/instance-admin/statistics — reads aggregate statistics from the database and reveals toggle count, user count, and project count. If the toggle count from the Admin API statistics endpoint differs significantly from the toggle count in the Client API snapshot, that's a data divergence signal: either the read replica is lagged or a bulk archive happened that the read replica hasn't reflected yet.
Flagsmith's evaluation health has a component that process probes miss entirely: the segment matching engine. Flagsmith's feature flag evaluation splits into two distinct paths. The first path is environment-level evaluation: GET /flags/ returns the environment-default state of every flag — no user identity involved, no segment matching, just a database read of the environment's feature state table. The second path is identity-level evaluation: POST /identities/ (or GET /identities/?identifier=xxx) retrieves flag states for a specific user, applying segment overrides if the user's traits match any segment rules. The segment matching engine is a separate evaluation component — it can produce incorrect results due to corrupted segment rules while the environment-level flag endpoint returns correctly.
The Flagsmith composite probe should include: (1) GET /health/ for process liveness; (2) GET /flags/ with the environment key to verify the flag store is readable and returns a consistent count; (3) POST /identities/ with a synthetic identity (__alivemcp_health_probe__ with no traits that should match any production segments) to exercise the segment matching engine. If probe identity evaluation returns flags that differ from environment defaults, that's a signal of unexpected segment matching state — the probe identity should match no production segments, so any override indicates a targeting rule with an unintended scope.
OpenFeature's evaluation health dimensions are unique because the abstraction layer creates new failure modes. OpenFeature.getProviderStatus() can return ProviderStatus.READY while evaluations return default values for one of two reasons: the targeting context is missing targetingKey (required by most providers for percentage rollouts and user targeting), or the flag key doesn't exist in the provider's backend (returns FLAG_NOT_FOUND with the default value). Neither of these returns an error that a *Value method caller can see — only the *Details variant exposes reason and errorCode fields that distinguish between a correctly-evaluated default and a silently-failing evaluation.
OpenFeature also introduces a dimension that doesn't exist in the direct-SDK model: provider status transitions. A provider in STALE status has lost its connection to the flag backend but retains its last cached configuration. Evaluations from a STALE provider return cached flag values, which may be minutes or hours old depending on how long the staleness has persisted. STALE is not ERROR — the provider is still functional, just not receiving updates. An OpenFeature health probe that only checks for status !== READY misses the STALE condition: set a warning threshold when STALE persists for more than 2 minutes, because flag updates from a recent split change are not propagating to running evaluations.
The synthetic evaluation probe is the most reliable OpenFeature health check: evaluate a flag key that doesn't exist in the provider's backend (__alivemcp_probe_flag__). A healthy provider returns errorCode: FLAG_NOT_FOUND. A provider with a connectivity problem to the flag backend returns errorCode: GENERAL or PROVIDER_NOT_READY. The distinction between FLAG_NOT_FOUND (probe succeeded, flag doesn't exist) and GENERAL (evaluation path is broken) is the health signal — and it requires getBooleanDetails(), not getBooleanValue(), to observe.
Split.io's evaluation health has an additional complexity: SDK readiness and streaming connectivity are independent dimensions. SDK_READY fires when the SDK has successfully downloaded a complete snapshot of all split definitions and segment memberships from Split.io's servers. Once ready, the SDK evaluates entirely from its in-memory cache — no network call per evaluation. But the streaming connection for real-time split updates can subsequently break, and the SDK silently falls back to polling on a 60-second interval. The SDK remains in READY status. Evaluations continue returning correct treatments from the cached definitions. But a split change made in the Split.io console doesn't reach the SDK for up to 60 seconds instead of the usual 5 seconds.
The Split.io health probe for production use requires verifying both dimensions: (1) SDK readiness via the SDK_READY event sequence during initialization; (2) an SDK_UPDATE event within a reasonable time window after a known split definition change (to verify the streaming connection is delivering updates). The second check is operational — it requires knowing that a change was made. For automated monitoring, the absence of SDK_UPDATE events for an extended period in an active development environment is a soft signal of streaming degradation.
Pattern 3: The Silent Impression Gap — Evaluation Succeeds but Experiment Data Corrupts
Feature flag evaluations produce two outputs. The first is the treatment or variation value returned to the application. The second is an impression or evaluation event — a record that flag X was evaluated for user Y with treatment Z, captured for experiment metric measurement. These two outputs travel through independent delivery channels. When the impression delivery channel fails, evaluations continue returning correct treatments (the application behaves correctly), but the experiment measurement backend stops receiving data. A/B test results show statistically impossible distributions, conversion metrics flatline on one treatment arm, or impression counts diverge 50/50 on paper while reality is 90/10 in traffic. The product team discovers it weeks later when an experiment run is analyzed.
The impression gap is the hardest failure mode to detect because it produces no observable error at the evaluation layer. HTTP 200, correct treatment value returned, evaluation latency normal — everything looks healthy. The damage is in the measurement pipeline that nobody is monitoring independently.
LaunchDarkly handles impressions through the Node.js SDK's event buffer. Every variationDetail() call generates an evaluation event that is buffered in memory and flushed to LaunchDarkly's Events endpoint on a 5-second interval (configurable via flushInterval). The event delivery channel is asynchronous and independent from the evaluation path. If the Events endpoint is unreachable — due to a network partition, endpoint deprecation, or certificate error — the SDK's event buffer fills, and eventually the buffer overflows and drops events silently. The evaluation path (reading from the in-memory flag store) continues unaffected.
Two practices protect against LaunchDarkly impression loss. First, always call await ldClient.flush() before process exit, before container shutdown, and before any planned maintenance window. The SDK's close() method attempts a flush, but in containerized environments with a 2-second SIGTERM-to-SIGKILL window, explicit pre-shutdown flushing is more reliable than relying on close(). Second, monitor experiment impression counts through the LaunchDarkly Events API or data export — a healthy experiment should show impression counts proportional to your traffic volume. Counts that are 10× lower than expected or completely absent from one treatment arm indicate impression delivery failure, not a traffic allocation problem.
Unleash records impression data as SDK clients report evaluation counts back to the Unleash server via the metrics reporting endpoint. The SDK polls for toggle configurations via the Client API and separately POSTs impression counts via /api/client/metrics. The Admin API's /api/admin/metrics/feature-toggles/raw endpoint exposes these aggregated counts per toggle, per environment, per application. If the metrics reporting endpoint is unreachable or the SDK's metrics buffer is being dropped, evaluation continues correctly (the SDK evaluates from its local cache regardless) but the impression counts go to zero.
Unleash's impression gap is detectable via the Admin API: if GET /api/admin/metrics/feature-toggles/raw returns zero counts for toggles that are known to be actively evaluated by production SDKs, the metrics delivery pipeline has stopped. Cross-reference the toggle's lastSeenAt field from GET /api/admin/features — if lastSeenAt has not advanced in the last 10 minutes during business hours, impression delivery has likely failed. Note: lastSeenAt is updated from the same metrics pipeline, so it fails in sync with impression counts — they're correlated signals, not independent.
There is also a variant management pitfall specific to Unleash that compounds impression gaps: variant weights sum to 1000 (per-mille), not 100 (percent). When updating variant configurations via the Admin API, a PATCH request where variant weights sum to anything other than 1000 returns a 400 error — but the error message sometimes says "Invalid strategy" rather than "Weight sum incorrect." An MCP tool that retries a failed variant update with the same weights repeatedly and receives opaque 400 errors may conclude the API is broken rather than that the weight arithmetic is wrong. Validate that variant weights sum to exactly 1000 before any PATCH to variant configurations.
Flagsmith has a unique impression-adjacent problem that affects probing rather than experiment measurement: POST /identities/ is not a stateless read. It creates or updates the identity and persists the provided traits in the database. Every call to the identities endpoint with a new identifier creates a new row. Every call with existing traits adds write pressure. An MCP tool that uses POST /identities/ as a health probe sends a write on every check interval — at a 60-second probe interval with 10 MCP servers, that's 10 writes per minute to Flagsmith's identities table. At scale, this probe load can degrade Flagsmith's write performance.
The correct probe approach for Flagsmith is: use GET /identities/?identifier=__alivemcp_health_probe__ for read-only health probing of existing identities; use POST /identities/ at lower frequency (or only on initial identity registration) to test the write path. The identity evaluation probe should use an identifier with no matching segment overrides — if the probe identity returns non-default flags, that's a targeting misconfiguration signal, not a health check pass.
Flagsmith also has a remote configuration dimension that other platforms lack: feature_state_value is an independent dimension from enabled. A flag can be "enabled" with a non-null value, or "disabled" with a non-null value — the value is stored regardless of the enabled state. An MCP tool that reads only enabled discards the configuration payload. An MCP tool that reads only feature_state_value misses the enable/disable gate. Both must always be read together, and feature_state_value is always a string regardless of the configured type — parse to the expected type explicitly (Number(rawValue) for numeric configs, JSON.parse(rawValue) for JSON blobs).
Split.io has the clearest separation between impression delivery and evaluation: getTreatment() and track() are entirely independent methods. getTreatment() evaluates a split from the in-memory cache and generates an impression event that's buffered and flushed to Split.io's servers. track() records a custom event (a conversion, a goal, a metric) that's also buffered and flushed separately. Both methods return silently when the SDK is not ready: getTreatment() returns "control", track() returns false. Neither throws. In both cases, no buffering occurs — the event is dropped on the floor.
The track() return value is the most commonly ignored health signal in Split.io integrations. Every MCP tool that records experiment conversion events should check const success = client.track(...) and log when success === false. At production scale, even a small percentage of dropped track() calls can shift experiment results: if the control arm's conversion events are being dropped at 5% while the treatment arm drops 0%, you'll observe false positive treatment lift. The success === false condition requires SDK not-ready state — if this appears in steady-state operation after SDK_READY fired, it indicates the client has become disconnected and needs reinitialization.
OpenFeature provides impression delivery infrastructure via hooks — the after() hook receives the evaluation result including value and reason, and the error() hook receives evaluation errors. These hooks can be used to ship impression data to any destination. But impression delivery in OpenFeature is fully provider-dependent: some providers (LaunchDarkly provider, Split.io provider) handle impression buffering and delivery internally; others leave it entirely to the application. There is no standard OpenFeature API for verifying that impressions are flowing — this is by design (OpenFeature doesn't mandate an impression model) but it means impression delivery verification must go through the underlying provider's native mechanism.
The OpenFeature pattern for impression monitoring is therefore: instrument hooks for local logging/metrics; then verify impression delivery through the underlying provider's API (LaunchDarkly event counts, Unleash metrics API, Split.io impressions in the data hub). The hook provides the signal that evaluations are happening; the provider's API provides the signal that those evaluations are generating records on the server side. Only together do they confirm the full end-to-end impression delivery path.
| Platform | Impression Delivery Mechanism | How It Fails Silently | Detection Method |
|---|---|---|---|
| LaunchDarkly | SDK in-memory event buffer → Events endpoint flush every 5s | Events endpoint unreachable; buffer overflows; events dropped without error | Monitor impression count via Events API or data export; call flush() before shutdown |
| Unleash | SDK metrics POST to /api/client/metrics after each poll cycle |
Metrics endpoint unreachable; lastSeenAt stops advancing |
GET /api/admin/metrics/feature-toggles/raw → zero counts = pipeline failure |
| Flagsmith | Provider-specific; identity evaluation is a write-per-call | POST /identities/ as probe adds write load; use GET /identities/ instead |
Cross-check identity eval vs environment eval; flag count baseline monitoring |
| OpenFeature | Provider-specific; hooks for local capture, provider for server-side delivery | Provider impression delivery invisible to OpenFeature SDK layer | Local hook counters + provider's native impression API |
| Split.io | SDK impression buffer; track() for custom events |
track() returns false silently when not ready; events not buffered |
Check track() return value; log all false returns in production |
Composite Probe Architecture for Feature Flag MCP Servers
Each of the three patterns above maps to a distinct probe tier. The credential validation tier confirms that credentials are valid before any flag operation. The evaluation health tier exercises the actual evaluation path end-to-end with a synthetic evaluation. The impression delivery tier verifies that evaluation events are reaching the server-side measurement backend. All three tiers must run independently — failure in any one tier is a distinct health problem with a distinct remediation path.
// Three-tier health probe architecture for feature flag MCP servers
async function featureFlagHealthProbe(platform, credentials) {
const results = await Promise.allSettled([
// Tier 1: Credential validation
// Confirm each credential type is valid and has the expected scope
validateCredentials(platform, credentials),
// Tier 2: Evaluation path health
// Exercise the actual evaluation path with a synthetic flag
// Healthy: returns FLAG_NOT_FOUND (LD/OpenFeature), "control" (Split.io),
// empty result (Flagsmith/Unleash) — not a system error
syntheticEvaluation(platform, credentials),
// Tier 3: Impression delivery health
// Verify that past evaluations have generated server-side impression records
// Check within the last 10-minute window for recently-active toggles
checkImpressionDelivery(platform, credentials),
]);
const [credTier, evalTier, impressionTier] = results;
return {
// Tier 1 failure = auth emergency (token revoked, rotated, or wrong type)
credentialsValid: credTier.status === 'fulfilled',
// Tier 2 failure = evaluation path broken (DB down, SDK not initialized)
evaluationHealthy: evalTier.status === 'fulfilled',
// Tier 3 failure = experiment data corrupting silently (no immediate user impact)
impressionDelivering: impressionTier.status === 'fulfilled',
// Alert routing:
// Tier 1 failure → page on-call immediately
// Tier 2 failure → page flag system owner
// Tier 3 failure → alert data/experimentation team (not on-call wake)
};
}
Alert routing is the key architectural decision: tier 3 impression delivery failures do not require an on-call wake. The application is behaving correctly — users are getting correct treatments. The data team needs to know that experiment data is corrupted so they can decide whether to extend the experiment run or invalidate the data set. Paging on-call for impression delivery failures creates alert fatigue because the immediate user impact is zero; routing to the data team creates accountability without urgency mismatch.
Probe frequency should also differ by tier. Tier 1 (credential validation) can run at a slow cadence — once per hour — since credentials are revoked rarely and the check is not a leading indicator of immediate failures. Tier 2 (evaluation path) should run at the standard 60-second interval, since a broken evaluation path immediately affects user experience. Tier 3 (impression delivery) should run at 5-minute intervals, checking whether impression counts have advanced in the last window — it doesn't need sub-minute resolution, and checking too frequently adds noise for a metric that's expected to be zero at low traffic times.
For AliveMCP monitoring of feature flag MCP servers, the three-tier architecture translates to three distinct probe configurations per flag system. The evaluation path probe is the primary — run at 60-second intervals with a 5-second timeout and page on failure. The credential probe runs hourly with a 15-second timeout and pages on 401/403 with a "credential rotation required" label. The impression delivery probe runs at 5-minute intervals, checks impression counts against a baseline, and routes to the data team's channel rather than on-call.
Cross-Platform Reference: Feature Flag MCP Integration Quick Reference
| Platform | Auth Model | Evaluation Method | Error Sentinel | Health Endpoint | Evaluation Reason | Open Source |
|---|---|---|---|---|---|---|
| LaunchDarkly | SDK key (streaming) + access token (REST) | SDK variationDetail(flagKey, ctx, default) |
reason.kind: ERROR + errorKind |
GET /healthcheck — process only |
TARGET_MATCH | RULE_MATCH | FALLTHROUGH | OFF | ERROR |
No (SaaS) |
| Unleash | Client token (no Bearer) + admin token (Bearer) | SDK isEnabled(toggleName, ctx) |
env.enabled=true + no strategies = disabled |
GET /health — process only |
Strategy name via impression label | Yes |
| Flagsmith | X-Environment-Key (env) + Api-Key (mgmt) |
GET /flags/ or POST /identities/ |
No throw — check enabled AND feature_state_value |
GET /health/ — process only |
Not exposed; derive from segment membership | Yes |
| OpenFeature | Provider-specific | client.getBooleanDetails(key, default, ctx) |
reason: ERROR + errorCode: GENERAL |
getProviderStatus() — provider status only |
STATIC | DEFAULT | TARGETING_MATCH | SPLIT | DISABLED | ERROR |
Yes (standard) |
| Split.io | SDK API key (streaming) + Admin API key (REST) | SDK getTreatmentWithConfig(key, splitName, attrs) |
treatment === "control" = always an error |
SDK_READY event — SDK sync only |
Impression label field (via impression listener) |
No (SaaS) |
The error sentinel column deserves emphasis: every platform has a different signal for "evaluation failed, here is the fallback." LaunchDarkly surfaces it in the reason object of variationDetail() — the variation value itself is the configured default, and the error is only in reason.kind === 'ERROR'. Unleash expresses errors structurally: a toggle that is environment-enabled but has no strategies evaluates to disabled — there's no error field, just a configuration state that produces unexpected behavior. Flagsmith has no error sentinel — a missing flag returns {found: false, enabled: false, value: null} which is structurally identical to a flag that exists and is deliberately disabled. OpenFeature standardizes on reason: ERROR plus errorCode, exposed only via *Details methods. Split.io uses "control" as the universal error sentinel — a treatment that should never appear in a correctly operating system.
The implication for MCP tool design: always use the evaluation detail variant that exposes the reason/error signal. variationDetail() not variation() in LaunchDarkly. getBooleanDetails() not getBooleanValue() in OpenFeature. getTreatmentWithConfig() with a check for treatment === "control" in Split.io. For Unleash and Flagsmith, validate the structural conditions (strategy count, enabled+value pair) before treating a result as a successful non-error evaluation. An LLM that asks "why did user X receive the control treatment?" deserves an answer that distinguishes between "user X was in the 50% rollout that received control" (correct evaluation) and "the SDK's streaming connection failed so every user got control" (evaluation system failure). Only the reason/error fields make that distinction possible.
What AliveMCP Monitors vs What Requires Internal Instrumentation
AliveMCP probes feature flag system health from the outside — it monitors the network-accessible health signals that confirm the system is operational. There are health dimensions that AliveMCP cannot monitor because they require access to internal SDK state or application-level metrics.
AliveMCP can monitor: REST API and Admin API credential validity (via credential verification endpoints); flag store population (via flag count from management API); evaluation path health (via synthetic flag evaluation against the management API or Client API); basic process liveness (via health endpoints); Client API toggle count vs Admin API toggle count divergence (data loss signal).
AliveMCP cannot monitor without internal instrumentation: SDK streaming connection quality (reconnect frequency, backoff state); per-SDK impression delivery buffer state; the gap between SDK_READY timing and actual request traffic timing; track() return value distributions in running applications; the difference between a flag returning its default because targeting rules selected it vs returning its default because of an error. These require application-side logging and metrics that feed into your own observability stack.
The practical division: AliveMCP alert = "your feature flag system's API layer is broken, act now." Internal instrumentation alert = "something subtle is wrong with how your application is using the feature flag system, investigate." Both are necessary. External monitoring catches outages; internal instrumentation catches misconfigurations and drift.
Related guides
- MCP tools for LaunchDarkly — flag evaluation, targeting rules, SDK vs REST API, and health monitoring
- MCP tools for Unleash — feature toggle evaluation, activation strategies, Admin vs Client API, and health monitoring
- MCP tools for Flagsmith — feature states, remote config, identity-based evaluation, and health monitoring
- MCP tools for OpenFeature — provider pattern, typed flag evaluation, hooks, and health monitoring
- MCP tools for Split.io — treatments, traffic allocation, dynamic config, impression tracking, and health monitoring
- MCP Tools for the Modern Data Stack — coordinator/worker split, background degradation, and health semantics inversion across Flink, Spark, dbt, ClickHouse, and Trino
- MCP server health check patterns — composite endpoints and failure classification
- MCP server uptime monitoring — probes, intervals, and alert routing