OpenShift Networking Deep Dive: Pods, Services, Routes, Policy and Egress

6 hours ago

16 min read

The architecture article drew networking as one layer in a larger platform. This one stays inside that layer. It follows a packet from a pod, across nodes, through a Service, in through the Ingress Controller, past network policy and out through an egress address, and it ties the main claims to Red Hat’s OpenShift Container Platform 4.22 documentation. Where a popular whiteboard diagram simplifies or blurs something, the text says so. The figures are redrawn, sharper originals that follow the whiteboard’s topics; they are not copies of it. The whiteboard itself appears in section 8; the other figures show each flow step by step.

1. The model in one page

OpenShift networking is a set of cooperating layers, each owned by an Operator or a plugin. The pod network and the Service network are virtual. They exist in software on every node and are carried over the node network that you or your cloud provide.

The default network provider is OVN-Kubernetes. Red Hat describes it as the default for OpenShift Container Platform and for single-node OpenShift, built on Open Virtual Network (OVN), with Open vSwitch (OVS) running on every node (About the OVN-Kubernetes network plugin). OVN describes the network in logical terms: logical switches, logical routers, ports and access control lists. OVS on each node then enforces that description as OpenFlow rules.

Pod network: each pod on the default network, other than host-network pods, gets an IP from a per-node slice of the cluster network CIDR. Pods can reach each other directly by those IPs, without NAT, regardless of node (Understanding networking).
Service network: a normal Service gets a stable virtual IP, the ClusterIP, from the service CIDR. Headless Services (clusterIP: None) have no ClusterIP, and ExternalName Services are a DNS alias with no proxying involved (Service API).
Ingress: HAProxy-based Ingress Controllers, managed by the Ingress Operator, accept external HTTP and HTTPS and send it to pods.
Policy: NetworkPolicy, AdminNetworkPolicy and BaselineAdminNetworkPolicy decide who may talk to whom.
Egress: EgressIP, egress firewall and egress router control how traffic leaves.

Everything else in this article hangs off those five ideas.

2. Who programs the network

Before looking at packets, look at the people doing the programming. The 4.22 architecture chapter (OVN-Kubernetes architecture) names the pieces and where they run, in the namespace openshift-ovn-kubernetes.

Figure 1. Original diagram drawn for this article from the OpenShift 4.22 OVN-Kubernetes architecture chapter. Not an official Red Hat diagram.

ovnkube-controller watches the Kubernetes API for pods, namespaces, Services, endpoints, egress IPs, egress firewalls and network policies. It also allocates pod IPs from the node’s subnet.
nbdb, the northbound database, holds the intended logical network: logical ports, switches and routers.
northd translates that logical configuration into logical data path flows in sbdb, the southbound database, which also holds chassis information and the binding tables.
ovn-controller reads the logical flows, turns them into OpenFlow rules and programs the node’s OVS.
ovnkube-control-plane runs as two pods on separate control plane nodes, one of them leader. Its ovnkube-cluster-manager container hands each node a pod subnet, a transit-switch IP and a join-switch IP.

Version note. The 4.22 chapter says the OVN northd, northbound database and southbound database run on each node and mostly hold information local to that node. Older write-ups, and older diagrams, often show one central pair of databases on the control plane. If you read a blog or a talk, check which release it describes.

The practical lesson is about failure domains. Because each node carries its own OVN databases and daemon, a problem on one node’s ovnkube-node pod mostly affects that node. Cluster-wide address planning sits with the control plane pods. When you troubleshoot, first decide whether the symptom is per node or cluster wide.

oc get all,ep,cm -n openshift-ovn-kubernetes
oc get pods -n openshift-ovn-kubernetes -o wide

The first command is the one Red Hat’s chapter uses to list the resources. The second adds the node each pod runs on, which is usually the next question.

3. The pod network and CIDR planning

The pod network is an overlay. Red Hat states that the plugin uses the Generic Network Virtualization Encapsulation (Geneve) protocol, rather than VXLAN, to build the overlay between nodes. Geneve traffic uses UDP port 6081, which appears in Red Hat’s firewall and port documentation for the cluster nodes.

Figure 2. Original diagram drawn for this article. Pod A to Pod B, step by step: the packet crosses the underlay inside Geneve; the underlay never sees pod IPs. Addresses are illustrative, using the documented default ranges. Not an official Red Hat diagram.

Follow the figure step by step. A pod sends a packet to another pod’s IP. OVN’s logical flows on the source node know which node owns the destination pod subnet. The packet is wrapped in Geneve and sent from the source node IP to the destination node IP. The destination node unwraps it and delivers it. To the physical network, this is ordinary UDP between two hosts. That is why the underlay does not need routes for pod subnets.

The default ranges

Red Hat’s CIDR chapter (CIDR range definitions) lists the defaults. Treat them as defaults, not as a design.

Machine CIDR: the documented default is 10.0.0.0/16, but it depends on the platform. The OpenStack installation parameters page, for example, says libvirt and some IBM platforms use different defaults (install-config parameters, OpenStack). Check the page for your platform. It cannot be changed after you create the cluster.
Pod CIDR (the cluster network): 10.128.0.0/14. You can expand it after installation.
Host prefix: /23, so each node gets a /23 slice of the pod CIDR.
Service CIDR: 172.30.0.0/16.
Reserved for OVN-Kubernetes internals: join 100.64.0.0/16, transit switch 100.88.0.0/16, and the masquerade subnet 169.254.0.0/17 for clusters on 4.17 and later. IPv6 equivalents exist.

A quick sizing exercise shows what the host prefix controls. Carving /23 slices out of a /14 gives 2 to the power of 9, which is 512 node slices, and each slice has 512 addresses (Red Hat’s installation docs describe a /23 as 510 usable pod addresses). A longer prefix such as /24 would give twice as many node slices, 1,024, but only half the addresses per node, 256. A shorter prefix does the reverse. Red Hat’s own worked example uses a /16 pod CIDR with a /23 host prefix and arrives at 128 nodes. Address capacity is not the same as supported pod density. How many pods a node may run is a separate limit, so size the prefix for the addresses you need and check the pod limits separately. This arithmetic is mine, based on the documented parameters. Use Red Hat’s OpenShift Network Calculator, linked from the CIDR chapter, for a real design.

Overlap is the classic mistake. The documentation says the join, transit and masquerade ranges must not appear in any other CIDR definition in your cluster or infrastructure. Pod and service ranges must also not overlap external services that pods call. Overlaps rarely fail loudly. They show up as pods that can reach everything except one subnet.

MTU: the cost of the overlay

Encapsulation adds bytes to every packet. Red Hat’s MTU chapter (Changing the MTU for the cluster network) says the cluster network MTU is always lower than the hardware MTU to make room for the overlay overhead. It also notes that changing the cluster MTU after installation is disruptive, because nodes must be rebooted. Decide your underlay MTU before you install. Jumbo frames on the underlay give you more room, but only if every switch and NIC on the path agrees.

4. Services, load balancing and DNS

Figure 3. Original diagram drawn for this article from the OpenShift 4.22 networking and DNS documentation. East-west traffic: DNS answers first, then the client connects to the ClusterIP and OVN picks a backend. Not an official Red Hat diagram.

Figure 4. Original diagram drawn for this article from the OpenShift 4.22 ingress documentation. North-south traffic: the user reaches the ingress VIP, the router matches the Route and proxies to a pod. Not an official Red Hat diagram.

Figure 3 shows the east-west path. A client pod looks up a name, gets a ClusterIP, and connects to it. Nothing listens on that IP. It is a virtual address that the network rewrites to a healthy backend.

DNS

The DNS Operator deploys CoreDNS as a daemon set, creates a Service for it, and configures the kubelet so pods use that Service IP for name resolution. It resolves internal cluster.local names and enables DNS-based Service discovery (DNS Operator in OpenShift Container Platform). In Red Hat’s example output the DNS Service has cluster IP 172.30.0.10, which is the tenth address of the default service CIDR. You can add forwarding for specific zones or upstream resolvers through the dns.operator/default resource, for example to send a corporate domain to an internal resolver.

How a Service is implemented

The whiteboard says kube-proxy load balances Service traffic in OVN. Red Hat’s architecture chapter describes it differently. When it lists the OVN northbound load balancers, it states that load balancers in OVN-Kubernetes are representations of Services, and the sample output maps Service IPs such as 172.30.0.1:443 to backend addresses (OVN-Kubernetes architecture). In other words, with the default provider the Service is realised inside OVN-Kubernetes as an OVN load balancer, not through a separate per-node kube-proxy process. The wording “kube-proxy (OVN)” mixes two different mechanisms. The behaviour is still what the whiteboard says: the client connects to the ClusterIP and the platform picks a backend.

Check, do not assume. Network provider internals change between releases. If you depend on how Service load balancing is implemented, for example for troubleshooting or performance work, read the chapter for your exact version.

5. Getting traffic in: Ingress Controller, Routes and load balancers

The Ingress Operator deploys and manages one or more HAProxy-based Ingress Controllers, and it handles both OpenShift Route objects and Kubernetes Ingress objects (Ingress Operator in OpenShift Container Platform). Every new installation has an IngressController named default. If you delete it, the Operator recreates it within about a minute. The default replica count is 2. The cluster Ingress configuration supplies the domain, for example apps.example.com, which also becomes the default host suffix for Routes that do not name a host.

Figure 4 shows the north-south path. An external client resolves a name under the apps domain. DNS points at a virtual IP or a load balancer in front of the router pods. The router matches the request to a Route using the host name, and for HTTP and edge or re-encrypt TLS also the path, then forwards to the pods behind the Route’s Service.

Route versus hop. Many diagrams draw Route, then Service ClusterIP, then pods, as if a packet visits each box. A Route is configuration. It names a Service as its target, and the router is an HTTP(S)-aware load balancing and proxy layer that serves it (Route API). Red Hat’s current chapters do not spell out the exact last hop the router takes, so this article does not either. Treat the ClusterIP path (east-west) and the Route path (north-south) as two separate ways to reach the pods behind the same Service.

TLS termination choices

Edge: the router decrypts, so it can read the host and path, then sends plain traffic to the pod. Certificates live on the Route or the router default (Routes).
Re-encrypt: the router decrypts, so it can read the host and path, then opens a new TLS connection to the pod. You supply the destination CA so the router can verify the pod’s certificate.
Passthrough: the router never decrypts. The pod holds the certificate. Because the router cannot read the request, path-based routing is not available.

A Route can also set insecureEdgeTerminationPolicy, for example Redirect to send plain HTTP to HTTPS. The default TLS security profile for the Ingress Controller is Intermediate, with a minimum of TLS 1.2, and the HAProxy image supports TLS 1.3 and the Modern profile.

Controlling who may claim a host name

The IngressController routeAdmission setting controls how Routes in different namespaces may claim the same host name. The default is Strict, which does not allow routes to claim the same host name across namespaces. This protects one team’s host from being hijacked by another team’s Route.

Publishing the router itself and other ways in

On cloud platforms the Ingress Controller is usually published through a load balancer Service. The endpointPublishingStrategy field also supports NodePort and host network strategies, which suit bare metal or restricted environments (Configuring ingress cluster traffic). For non-HTTP workloads, a Service of type LoadBalancer works with a cloud load balancer or, on bare metal, with MetalLB, which allocates addresses from address pools and advertises them with layer 2 or BGP (Load balancing with MetalLB). NodePort Services use ports 30000 to 32767. Red Hat’s 4.22 documentation also has a chapter on configuring Gateway API (Configuring Gateway API). It is worth reading if you plan new ingress designs.

6. Network policy: who may talk to whom

Figure 5. Original diagram drawn for this article from the OpenShift 4.22 network security chapter. Policy evaluation moves down the tiers and stops at the first tier that decides the connection. Not an official Red Hat diagram.

Out of the box, pods are open. Red Hat states that by default all pods in a project are accessible from other pods and network endpoints, and a pod not selected by any NetworkPolicy remains fully accessible (Network policy). Once any NetworkPolicy selects a pod, that pod accepts only connections that at least one policy allows. Policies are additive.

4.22 adds a cluster-wide layer. Understand network policy APIs describes three tiers, and OVN-Kubernetes evaluates them in this order:

AdminNetworkPolicy (cluster administrators). Rules are Allow, Deny or Pass. Allow and Deny decide immediately and NetworkPolicy cannot override them. Pass hands the decision to the NetworkPolicy tier.
NetworkPolicy (namespace owners). It works per direction. A pod selected by a policy for ingress, or for egress, becomes isolated in that direction: Red Hat’s comparison table describes this as implicit deny isolation at policy creation time. Connections that no selecting policy allows are dropped in this tier. The BaselineAdminNetworkPolicy does not get a second chance to allow them.
BaselineAdminNetworkPolicy (cluster administrators, a singleton named default). It is the last tier, with Allow and Deny rules only. It applies to traffic not decided above, for example traffic an AdminNetworkPolicy passed where no NetworkPolicy selects the pod. Red Hat describes it as a guardrail, and as a way to make intra-cluster traffic blocked by default so teams must write NetworkPolicy to allow what they need.

Two details are easy to miss. First, direction is independent. A connection from pod A to pod B has to be acceptable on A’s egress side and on B’s ingress side. If pod B has an ingress policy that does not allow A, the connection fails even if A has no egress policy, and the reverse also holds. Second, an AdminNetworkPolicy and a NetworkPolicy can both apply to the same pod, in different tiers, as the evaluation order above shows.

What this corrects on the whiteboard

The whiteboard says NetworkPolicy has allow and deny rules and implies that traffic between namespaces is blocked. Neither is quite right. Standard NetworkPolicy lists what is allowed, and selecting a pod isolates it in that direction. Explicit Deny exists in AdminNetworkPolicy. Traffic between namespaces is allowed until you add a default-deny policy plus the allow rules you need, or an AdminNetworkPolicy or BaselineAdminNetworkPolicy. The statement about a distributed firewall is fair: the rules become OVN access control lists on each node, enforced through OVS.

Practical patterns

Start with a default-deny policy per namespace, then add allow rules. Red Hat’s example blocks all cross-pod traffic except what other policies allow and traffic between host-networked pods.
Allow traffic from the Ingress Controller to your application pods explicitly. Red Hat’s documentation shows an allow-from-openshift-ingress policy for this.
Keep policies few and broad. Red Hat notes that each rule based on a pod or namespace selector produces OVS flows in proportion to the pods selected, so merging policies with the same pod selector is more efficient.

Platform namespaces. The DNS and Ingress Operators install deny-all policies in their own namespaces, with narrow egress. From 4.22 some other OpenShift namespaces also include NetworkPolicy objects by default. Do not edit them, and do not run unmanaged pods in those namespaces, because traffic for them will be blocked.

7. Getting traffic out: gateway, EgressIP, firewall and router

Outbound traffic raises different questions. Which address does the external system see? May this workload call that destination at all? Should traffic leave through the node, or through a dedicated gateway?

Figure 6. Original diagram drawn for this article. The pod’s traffic leaves through the node holding the EgressIP, so the firewall and the database see one stable address. Addresses follow the documentation style. Not an official Red Hat diagram.

EgressIP

With OVN-Kubernetes you can assign one or more egress IP addresses to a namespace, or to specific pods in it, so that traffic to services outside the cluster network has a consistent source IP (Configuring an egress IP address). Typical use: an external database allows only a known address, and you want only one workload to use it.

Label the nodes that may hold egress IPs with k8s.ovn.org/egress-assignable="". At least one labeled node is required.
Label before you create EgressIP objects. Otherwise OpenShift might assign every address to the first labeled node.
An egress IP is never assigned to more than one node at a time, and no node hosts more than one of the addresses listed in the same EgressIP object. Addresses are balanced between the nodes that can host them.
If an EgressIP object lists more addresses than you have egress-assignable nodes, the extra addresses stay unassigned. Keep the node count at least equal to the address count.
Traffic from the selected pods is balanced roughly equally between the listed addresses. Red Hat says there is no guarantee which one is used: either may be used for each TCP connection or UDP conversation. So the external allow-list must contain every address in the object.
If a pod matches the selector of more than one EgressIP object, there is no guarantee which object’s address it gets.
Egress IP traffic leaves through the interface where the address is assigned. Never configure these addresses in Linux network configuration files.
If a node becomes unavailable, its addresses are reassigned automatically under the same rules. Failover is health-checked. The reachabilityTotalTimeoutSeconds setting controls how long the platform waits before declaring a node unreachable, and Red Hat says to expect failover on the order of seconds or longer.
oc get egressip shows only one address however many are configured. Use -o yaml to see them all.
If a pod and the router pod are on the same node, the egress IP is not used when that pod calls a Route.

The whiteboard calls this an “Egress NAT gateway”. That is not an OpenShift feature name. The documented mechanism is EgressIP, which gives the selected pods an assigned source address.

Egress firewall and egress router

An egress firewall (Egress firewall) decides which external destinations selected pods may reach. It does not change the source address. An egress router redirects traffic to specific destinations and is a different tool from an egress IP. Red Hat lists egress firewall, egress IP and egress router in redirect mode among the supported OVN-Kubernetes capabilities. A common layered design is EgressIP for a stable source address plus an egress firewall for allowed destinations.

Gateway mode: through the host or not

The Cluster Network Operator’s gatewayConfig.routingViaHost setting (Configuring a gateway) controls how egress traffic leaves. With true, traffic goes through the node’s local gateway and follows the host routing table, which helps when you need kernel routes or an external NAT device. With false, OVS sends traffic out of the node IP interface directly. Choose it deliberately and before you have workloads that depend on a particular path.

8. Whiteboard claims, checked

These are the specific statements from the supplied diagram that needed care, with the Red Hat evidence for each. The diagram is a good learning aid. This table is about precision.

The supplied whiteboard, shown at its original 1536 x 1024 resolution. The redrawn figures in this article carry the detail; the table below checks its claims. Not official Red Hat documentation.

Whiteboard claim	Verdict against Red Hat 4.22 docs
OVN-Kubernetes is the default pod network	Correct. Default for OpenShift Container Platform and single-node OpenShift in 4.22.
Pod CIDR 10.128.0.0/14, service CIDR 172.30.0.0/16	These are documented defaults, not fixed values. The pod CIDR can be expanded after installation. The machine CIDR default depends on the platform and cannot be changed after installation.
Geneve tunnels between nodes	Correct. Red Hat documents Geneve, not VXLAN, and UDP port 6081.
“kube-proxy (OVN)” balances Services	Imprecise. Red Hat’s chapter shows Services as OVN load balancers. Do not read this as a per-node kube-proxy.
Routes are built on HAProxy	Correct. The Ingress Operator manages HAProxy-based Ingress Controllers. Route then ClusterIP then pod is a logical picture, not a confirmed hop-by-hop path.
TLS: edge, re-encrypt, passthrough	Correct. Passthrough rules out path-based routing.
NetworkPolicy has allow and deny rules	Imprecise. NetworkPolicy lists allowed traffic and isolates selected pods per direction. Explicit Deny is in AdminNetworkPolicy.
Default is allow all	Correct for pods no policy selects.
Egress NAT gateway	Not a documented OpenShift name. Use EgressIP, egress firewall and egress router terms.
EgressIP whitelists at the external firewall	Fair as a use case, but allow-list every address in the object: either may be used per connection. Check the same-node router caveat.
DNS: CoreDNS at the service IP	Correct. DNS Operator deploys CoreDNS as a daemon set with its own Service.
Pods never directly routable in the underlay	Mostly right for the overlay path. Pod IPs ride inside Geneve. External-bound traffic is translated, for example by EgressIP or the node address.

9. A troubleshooting order that follows the layers

When something cannot connect, walk the same layers in the same order as this article. Each step narrows the cause:

Name: does the name resolve from inside a pod? Check the DNS Operator status and the dns-default pods.
Address: does the Service have endpoints? An empty endpoint list means a selector or readiness problem, not a network problem.
Reachability: can the pod reach a pod on another node? If same-node works and cross-node fails, suspect the underlay, UDP 6081 or MTU.
Policy: is any NetworkPolicy or AdminNetworkPolicy selecting the destination pod? Remember that selecting a pod changes it from open to allow-listed.
Edge: for external access, check the IngressController status, the Route’s admitted condition and its TLS mode.
Egress: if an external system rejects the pod, check which source address it sees, then the EgressIP assignment and the egress firewall.

oc get clusteroperator network dns ingress
oc get ingresscontroller -n openshift-ingress-operator
oc get networkpolicy -A
oc get egressip -o yaml

Operator status first. The network, DNS and ingress Cluster Operators are the first stop when a symptom is cluster wide. A degraded Operator often names the cause.

10. Planning checklist

Choose machine, pod, service and any other CIDRs together. Keep them clear of join, transit and masquerade ranges, and of every network your pods will call.
Choose the host prefix from the number of nodes and the density you expect.
Decide underlay MTU before installation. Changing the cluster MTU later is disruptive.
Allow UDP 6081 between nodes, and review Red Hat’s firewall and port list for the other node-to-node flows.
Plan DNS: the apps wildcard record, the API record and any forwarding zones.
Decide how the default Ingress Controller is published, and whether sharded or additional controllers are needed.
Adopt a default-deny pattern for application namespaces, with an explicit allow from the Ingress Controller.
If external systems filter by IP, plan EgressIP nodes, labels and failover before go-live.
State the release you design against. This article is based on OpenShift Container Platform 4.22 documentation.

11. Where this fits in the series

The architecture article introduced networking next to storage, security and operations. This one expanded just the network layer. The same method applies to the rest of the platform: draw the layers, name the Operator that owns each, and verify each claim in the documentation for your version.

References

Diagrams in this article were drawn for it from the cited documentation. They are not official Red Hat diagrams. Based on OpenShift Container Platform 4.22 documentation, checked November 11, 2026.