A client needed several isolated OpenShift environments but had only a few servers and very few IP addresses. I designed a nested architecture that delivered separate clusters on the hardware they already owned, without buying more servers or requesting more IPs.
CLIENT
Enterprise client
ROLE
Design, build and handover
EMPLOYER
FABS
TECHNOLOGY
Red Hat OpenShiftOpenShift Virtualization (KubeVirt)Kubernetes Gateway APIOpenShift Data FoundationAssisted Installer
[HERO IMAGE: rack, console or OpenShift dashboard, sanitised]
3
isolated OpenShift clusters
0
additional IP addresses
context
The client wanted separate OpenShift clusters so teams could work in isolation, for example development, testing and production. A standard design would put each cluster on its own servers with its own set of IP addresses for the API, ingress and nodes, but they had neither: their hardware could support only one bare-metal cluster, and their network team could allocate just a handful of IP addresses. Buying more servers or redesigning the network was off the table, so the solution had to fit what was already there.
hardwareLimited physical servers, enough for one bare-metal cluster
networkVery few routable IP addresses available
isolationEach environment still needed proper isolation
accessWorkloads had to be reachable from outside the cluster
architecture
client network · users
bare-metal openshift
host cluster on the client's servers
Gateway API
1 external IP · routes by hostname
dev.apps
nested ocp · devcontrol plane · 3 VMsworkers · VMsapps pod network only
test.apps
nested ocp · testcontrol plane · 3 VMsworkers · VMsapps pod network only
prod.apps
nested ocp · prodcontrol plane · 3 VMsworkers · VMsapps pod network only
openshift virtualization · VMs run as pods
existing bare-metal servers · no new hardware
1one way in a single Gateway API entry point and one external IP serve every environment
2full isolation each team gets its own OpenShift cluster, API, console and access rules
3no extra IPs workloads stay on the pod network, so nested clusters need no routable addresses
4shared hardware all clusters run on the servers the client already owned
what i did
01Chose the install method.Used the OpenShift Assisted Installer for the bare-metal host cluster, then the same workflow inside each VM set to bring up the nested dev, test and prod clusters, so every install followed one repeatable process.
02Sized the virtual machines.Sized control plane and worker VMs per environment to the workload each team actually ran, rather than giving dev, test and prod identical capacity.
03Provisioned storage.Used OpenShift Data Foundation on the host cluster to provide persistent storage to the VMs and to the nested clusters running inside them.
04Set up Gateway API routing.Configured the Gateway API on the host cluster with hostname-based routes per environment, so dev.apps, test.apps and prod.apps all resolved through the same external IP.
05Enforced access and isolation.Gave each nested cluster its own API, console and RBAC, with network policies and namespace boundaries so no environment could reach another's workloads.
trade-offs
Nesting adds a layer of virtualisation, so it is best suited to development, testing and lab environments rather than the most performance-sensitive production workloads. Upgrades also need to be coordinated across both the host and the nested clusters. These were discussed with the client up front, so the design matched how they planned to use each environment.