Guide · EKS Network Security

Network policies for MCP servers on EKS

The most dangerous thing about Kubernetes NetworkPolicies is that you can write them, apply them, and watch kubectl get networkpolicy return them — while every pod in your cluster still freely talks to every other pod because the CNI plugin isn't enforcing anything.

TL;DR

The default EKS VPC CNI does not enforce NetworkPolicies — you must either install Calico/Cilium or enable the Amazon VPC CNI network policy controller (EKS add-on, v1.14+). Once enforcement is active, always include an explicit egress rule allowing UDP and TCP port 53 to kube-system or DNS resolution silently breaks for every pod the policy applies to. Start with a default-deny policy, then add allowlist rules for your API gateway namespace, monitoring namespace, and AWS service endpoints.

NetworkPolicy enforcement: the CNI requirement

A Kubernetes NetworkPolicy is a declaration of intent stored in etcd. Whether that intent is enforced depends entirely on the CNI plugin running on each node. The default EKS VPC CNI (aws-node DaemonSet) does not enforce NetworkPolicies — it focuses on VPC IP address management, not traffic filtering.

Your options for enforcement:

Option How to enable Notes
Amazon VPC CNI with network policy controller EKS add-on version v1.14+; set ENABLE_NETWORK_POLICY_CONTROLLER=true on aws-node Native AWS support; uses eBPF; initially same-node enforcement; no extra components to manage
Calico (CNI plugin or policy-only overlay) Install via Helm or operator; can run alongside VPC CNI in "policy-only" mode Mature, well-documented; GlobalNetworkPolicy extends standard K8s policies; adds CRDs
Cilium Replace VPC CNI with Cilium CNI (requires node restart); or run in chained mode eBPF-native; L7 HTTP/DNS filtering; best observability via Hubble; higher operational complexity

To verify enforcement is active after installing your CNI of choice, apply a test deny policy and confirm that traffic is actually blocked — don't rely on the policy being present in the API server.

# Quick enforcement test
# 1. Apply a deny-all NetworkPolicy to a test namespace
# 2. exec into a pod in that namespace
# 3. Try to curl another pod — it should fail
# 4. Remove the policy — it should succeed again

kubectl exec -n test-ns test-pod -- curl -s --max-time 3 http://other-pod.test-ns/
# Should timeout if enforcement is working

Default-deny pattern

The foundation of MCP server network isolation is a default-deny policy that blocks all ingress and egress within a namespace, followed by explicit allowlist policies. Apply this before you add any allowlist rules — but only to namespaces where you've already defined the allowlists you need, or you'll break running pods.

# default-deny-mcp.yaml
# Applies to ALL pods in the mcp namespace (empty podSelector = match all)
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
  namespace: mcp
spec:
  podSelector: {}      # matches every pod in the namespace
  policyTypes:
    - Ingress
    - Egress
  # No ingress or egress rules = deny everything

After applying this, no pod in the mcp namespace can send or receive any traffic — including DNS queries. You must add allowlist policies before your MCP pods will function. The order of operations matters: apply this last, after your allowlist policies are in place, or do it in a maintenance window.

DNS egress rule — you must not skip this

The most common misconfiguration when adding NetworkPolicies is forgetting the DNS egress rule. When you apply any egress NetworkPolicy to a pod, all egress traffic not explicitly allowed is dropped. DNS resolution uses UDP and TCP port 53 to kube-dns in the kube-system namespace. Without this rule, every hostname lookup fails silently — your MCP server can't reach its database, its LLM API, or any Kubernetes service by name.

# allow-dns-egress.yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-dns-egress
  namespace: mcp
spec:
  podSelector: {}        # applies to all pods in the namespace
  policyTypes:
    - Egress
  egress:
    - to:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: kube-system
      ports:
        - protocol: UDP
          port: 53
        - protocol: TCP
          port: 53        # TCP fallback for large DNS responses

Note: kubernetes.io/metadata.name is a label that Kubernetes automatically sets on every namespace to equal the namespace name. You don't need to add it manually — it exists on kube-system by default. This is what makes namespaceSelector.matchLabels.kubernetes.io/metadata.name work without any extra setup.

MCP server allowlist policy

With default-deny and DNS egress in place, add the specific allowlist rules for your MCP server. This example allows ingress from the API gateway namespace and the monitoring namespace, plus egress to external LLM APIs and AWS service endpoints.

First, label your namespaces — namespaceSelector matches on labels, not namespace names:

# Label namespaces for NetworkPolicy selectors
kubectl label namespace api-gateway  role=api-gateway
kubectl label namespace monitoring   role=monitoring
kubectl label namespace mcp          role=mcp-server

# Verify
kubectl get namespace api-gateway -o jsonpath='{.metadata.labels}'

Now the MCP server allowlist policy:

# mcp-server-networkpolicy.yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: mcp-server-allowlist
  namespace: mcp
spec:
  podSelector:
    matchLabels:
      app: mcp-server      # only applies to mcp-server pods
  policyTypes:
    - Ingress
    - Egress
  ingress:
    # Allow traffic from the API gateway namespace
    - from:
        - namespaceSelector:
            matchLabels:
              role: api-gateway
      ports:
        - protocol: TCP
          port: 8080        # your MCP server's port
    # Allow Prometheus scraping from the monitoring namespace
    - from:
        - namespaceSelector:
            matchLabels:
              role: monitoring
      ports:
        - protocol: TCP
          port: 9090        # your metrics port
  egress:
    # Allow HTTPS to external LLM APIs and AWS services
    # You cannot use hostnames in ipBlock — must use CIDRs
    # For AWS APIs, use VPC endpoints to avoid external CIDR management
    - ports:
        - protocol: TCP
          port: 443
      to:
        - ipBlock:
            cidr: 0.0.0.0/0
            except:
              - 10.0.0.0/8       # block egress back into the VPC except...
              - 172.16.0.0/12
              - 192.168.0.0/16
    # Allow egress to in-cluster services (database, cache, etc.)
    - to:
        - namespaceSelector:
            matchLabels:
              role: data-layer
      ports:
        - protocol: TCP
          port: 5432         # PostgreSQL
        - protocol: TCP
          port: 6379         # Redis

AWS service endpoint caveat: AWS SDK calls to services like S3, Secrets Manager, and STS go to AWS API endpoints. The easiest pattern is to create VPC Interface Endpoints for each AWS service you use — traffic stays within the VPC and uses private IPs, so your egress rules covering internal CIDR ranges work. Without VPC endpoints, you need to allow egress to AWS public IP ranges, which are large and change periodically.

Multi-tenant MCP namespace isolation

If you run multiple tenants on the same EKS cluster with one MCP namespace per tenant, apply default-deny to each namespace and use namespace labels to control cross-tenant traffic explicitly.

# Multi-tenant setup: per-tenant namespace with default deny + selective allows
# Repeat this pattern for each tenant namespace

---
# 1. Label the tenant namespace
# kubectl label namespace tenant-acme tenant=acme

# 2. Default deny in each tenant namespace
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
  namespace: tenant-acme
spec:
  podSelector: {}
  policyTypes:
    - Ingress
    - Egress

---
# 3. DNS egress (same pattern as above, applied per namespace)
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-dns-egress
  namespace: tenant-acme
spec:
  podSelector: {}
  policyTypes:
    - Egress
  egress:
    - to:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: kube-system
      ports:
        - protocol: UDP
          port: 53
        - protocol: TCP
          port: 53

---
# 4. Allow only the shared API gateway to reach this tenant's MCP server
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-gateway-ingress
  namespace: tenant-acme
spec:
  podSelector:
    matchLabels:
      app: mcp-server
  policyTypes:
    - Ingress
  ingress:
    - from:
        - namespaceSelector:
            matchLabels:
              role: api-gateway
          podSelector:
            matchLabels:
              tenant: acme       # further restrict to tenant-specific gateway pods
      ports:
        - protocol: TCP
          port: 8080

Note: namespaceSelector and podSelector inside the same from entry are ANDed — the pod must be in a matching namespace AND have matching pod labels. If you put them in separate from list items they are ORed. This distinction causes subtle security bugs — always verify with an enforcement test after writing complex policies.

Failure modes reference

Symptom Root cause Fix
NetworkPolicy applied but traffic still flows between all pods CNI plugin does not enforce NetworkPolicies (default VPC CNI with no policy controller) Install Calico, Cilium, or enable Amazon VPC CNI network policy controller; verify with an active enforcement test
All DNS resolution fails after applying egress policy Missing DNS egress rule — port 53 UDP/TCP to kube-system is blocked Apply the allow-dns-egress NetworkPolicy; check with kubectl exec -- nslookup kubernetes.default
Traffic blocked despite namespaceSelector looking correct Target namespace doesn't have the required label; namespaceSelector matches labels, not names Run kubectl get namespace <name> --show-labels; add the label with kubectl label namespace <name> role=<value>
AWS SDK calls fail (S3, Secrets Manager, STS) after adding egress policy Egress to AWS public API endpoints blocked; no VPC endpoint, so traffic leaves the VPC Create VPC Interface Endpoints for each AWS service in use; alternatively add egress rule allowing port 443 to AWS IP ranges (less recommended)
New pods can't initialize — crash-looping immediately after namespace default-deny is applied Default-deny applied before allowlist policies; pods can't reach their dependencies on startup Apply allowlist policies first; apply default-deny last; or use a maintenance window to apply all policies atomically
namespaceSelector + podSelector in same from entry blocks expected traffic AND semantics: pod must match both selectors simultaneously; works differently than two separate from list entries Test each policy with kubectl exec; use separate from entries when OR semantics are needed; document intent in policy annotations
Prometheus scraping fails after adding ingress deny No ingress rule for the monitoring namespace on the metrics port Add ingress rule from namespaceSelector: role=monitoring on your metrics port (default 9090); verify with kubectl exec -n monitoring -- curl http://mcp-pod-ip:9090/metrics