OpenShift AI on GPU servers: GPU Operator, RTX PRO Server Edition and H200 NVL workers

What an OpenShift AI GPU node needs

A GPU worker node for Red Hat OpenShift AI needs three things from the NVIDIA side: a card on the GPU Operator's support list, the NVIDIA GPU Operator installed through the Operator Lifecycle Manager (OLM) with a ClusterPolicy, and, for NVIDIA NIM or other AI Enterprise software, an NVIDIA AI Enterprise licence for each GPU. In October 2026 the current release is OpenShift AI Self-Managed 3.5, and NVIDIA's platform page supports OpenShift 4.18 to 4.22 for GPU Operator 26.7.x.

Everything else is ordinary OpenShift: the node joins the cluster, the Node Feature Discovery Operator labels it, the GPU Operator runs the NVIDIA driver as a container and the device plugin advertises nvidia.com/gpu. OpenShift AI then sees the GPUs through hardware profiles. This article lists the requirements and the server sizes; our guide to a private LLM platform on Kubernetes covers the serving stack on upstream Kubernetes, and VMware Private AI Foundation with NVIDIA the VCF route.

NVIDIA GPU Operator platform support page (14 October 2026), NVIDIA AI Enterprise 8.2 support matrix (2 September 2026), Red Hat OpenShift AI Self-Managed 3.5 documentation; all read on 24 October 2026.

Which cards OpenShift AI worker nodes can use

The GPU Operator's support list is the first filter. It names, among the cards we supply, the RTX PRO 6000 Blackwell Server Edition, the RTX PRO 4500 Blackwell Server Edition, the H200 NVL, the L40S and the L4. For the RTX PRO 6000 Server Edition it requires driver 575.57.08 or later and notes that MIG is not supported on 575.57.08 itself. We found no entry for the RTX PRO 6000 Workstation or Max-Q editions or for the RTX PRO 5000, so workstation cards do not belong in OpenShift worker nodes.

The second filter is the serving software. Red Hat's supported configurations for Red Hat AI 3 list the NVIDIA accelerators for Red Hat AI Inference Server, Red Hat's vLLM distribution: among others the L4 and L40S, the H100 and H200, and "RTX PRO 6000 Blackwell Server Edition" and "RTX PRO 4500 Blackwell Server Edition". The H200 NVL does not appear under its own name; the list has the H200. On a node with H200 NVL cards, have Red Hat confirm that the NVL form is covered before you rely on that support.

NVIDIA GPU Operator platform support page (14 October 2026); Red Hat AI 3 supported product and hardware configurations (AI Inference Server 3.4.0-ea.1); MIG from NVIDIA's MIG user guide of 13 October 2026, as in our MIG guides; all read on 24 October 2026.

NVIDIA's platform page also names the GPUs validated for OpenShift on the large public clouds, from the T4 to the H200, and does not name the RTX PRO cards there. Both RTX PRO Server Editions are on the Operator's general GPU list and on Red Hat's accelerator list. We supply all five cards; tell us which OpenShift release your cluster runs, and we propose worker nodes whose cards are on both lists.

Installing the GPU Operator through OLM

NVIDIA's OpenShift guide, updated 14 October 2026, installs the certified operator from the console catalog: Ecosystem, then System Catalog on 4.20 and later, Operators, then OperatorHub before. The suggested namespace is nvidia-gpu-operator; with another namespace, monitoring is not enabled automatically. On the command line the steps are a namespace, an OperatorGroup, a Subscription to the package gpu-operator-certified with installPlanApproval: Manual, and approval of the install plan. Red Hat's note for 4.21 and later: the OLMv1 catalog is enabled by default as a Technology Preview, and you disable it to get the standard installation form back.

Installing the operator creates the ClusterPolicy definition. NVIDIA's guide takes the default instance, gpu-cluster-policy, from the alm-examples annotation of the ClusterServiceVersion and warns that a ClusterPolicy with an empty spec "fails to deploy". Allow 10 to 20 minutes before troubleshooting; success shows State:ready.

Three points matter at upgrades. NVIDIA's platform page marks 26.7.x as supported and 26.3.x as deprecated, while the CLI example in the OpenShift guide still subscribes to channel 26.3; read the channels from the packagemanifest and choose a supported one. As of 23 October 2026 our guide to GPU sharing in Kubernetes advises 26.7.2 or later, since releases up to 26.3.1 are affected by CVE-2026-24260. And OLM does not update custom resources when the operator is upgraded: NVIDIA's notes for 25.10.0 recommend setting cdi.enabled to true in the ClusterPolicy after that upgrade for this reason. After every Operator upgrade, compare the ClusterPolicy with the defaults of the new release.

GPU Operator 26.7 also offers Dynamic Resource Allocation through a GPUCluster resource in place of the ClusterPolicy, and NVIDIA has added a DRA installation guide for OpenShift. A cluster uses one or the other. Red Hat's procedure for OpenShift AI creates a ClusterPolicy, so for OpenShift AI we stay with that; our guide to Kubernetes DRA for NVIDIA GPUs explains the alternative.

Enabling the GPUs in OpenShift AI

Red Hat's OpenShift AI 3.5 guide "Enable NVIDIA GPUs" assumes cluster-admin rights and an installed GPU that the environment detects. The procedure:

1. Install the Node Feature Discovery Operator and create a NodeFeatureDiscovery instance.
2. Install the NVIDIA GPU Operator and create a ClusterPolicy with default values.
3. Delete the ConfigMap migration-gpu-status in the project redhat-ods-applications.
4. Restart the rollout of the deployment rhods-dashboard and wait until all pods have restarted.
5. Check with oc describe node that the node lists nvidia.com/gpu under Capacity and Allocatable.

Red Hat adds two limits: it supports accelerators "within the same cluster only", and since OpenShift AI 2.19 RDMA is supported for NVIDIA GPUs only. The Installed Operators page should then show the NVIDIA GPU Operator, Node Feature Discovery and Kernel Module Management.

Hardware profiles connect users to the nodes. An administrator creates them under Settings, Environment setup, Hardware profiles. Each profile carries resource entries with a default, minimum and maximum, node selectors, tolerations and, when Kueue manages scheduling, a local queue. A profile for the language model's serving runtime selects the H200 NVL or RTX PRO 6000 Server Edition nodes by label and tolerates their taint; a second profile for notebooks can point at L4 nodes. The labels come from GPU Feature Discovery, for example the GPU model in nvidia.com/gpu.product.

MIG and time-slicing on OpenShift worker nodes

MIG on OpenShift follows NVIDIA's OpenShift guide: patch the ClusterPolicy at /spec/mig/strategy to single or mixed, then label the node with nvidia.com/mig.config and a layout; all-disabled is the default. MIG Manager discovers the profiles of each GPU and writes a ConfigMap per node. The node must be drained of GPU workloads first: MIG Manager terminates GPU pods itself during the change, and if the node is not drained the state label nvidia.com/mig.config.state reads failed.

With the single strategy, all GPUs of a node use the same layout and slices appear as nvidia.com/gpu. With mixed, each profile is its own resource, so a quarter of an RTX PRO 6000 appears as nvidia.com/mig-1g.24gb and the smallest H200 NVL slice as nvidia.com/mig-1g.18gb. A hardware profile then names that resource as its identifier, by our reading of Red Hat's profile fields. A model split over several GPUs needs whole cards, because NCCL is not supported with MIG; our RTX PRO 6000 MIG profiles and H200 NVL MIG profiles show what fits in each slice.

Time-slicing is Red Hat's other documented method. A ConfigMap time-slicing-config in the nvidia-gpu-operator namespace sets a replicas count per GPU model, the ClusterPolicy's devicePlugin.config points to it, and the machine set labels the nodes with nvidia.com/device-plugin.config. Red Hat states that "All workloads share GPU memory" and that time slicing gives no strict isolation, so it suits notebooks and light inference, and MIG suits shared production endpoints.

Model serving: vLLM runtime or NVIDIA NIM

OpenShift AI 3.5 preinstalls one serving runtime for NVIDIA GPUs, the vLLM NVIDIA GPU ServingRuntime for KServe. Red Hat lists the NVIDIA Triton Inference Server among tested and verified runtimes that an administrator adds by hand.

The NVIDIA NIM model serving platform is enabled from the Applications, Explore page of the OpenShift AI dashboard. Red Hat's prerequisites: GPU support enabled, an NVIDIA Cloud Account with access to the NGC portal, that account "associated with the NVIDIA AI Enterprise Viewer role", and a personal API key generated on NGC, entered on the NIM tile. After an OpenShift AI upgrade the key has to be entered again. The entitlement therefore comes from NVIDIA, not from the Red Hat subscription, as our guide to NVIDIA AI Enterprise licensing explains: NVIDIA's NIM FAQ requires an AI Enterprise licence for NIM in production, counted per GPU, and each H200 NVL includes a five-year AI Enterprise subscription. Our article on activating NVIDIA AI Enterprise licences covers the NGC organisation and keys.

Red Hat's AI Inference Server images in that list are built with CUDA 13.0 and are, in Red Hat's words, "backward compatible with CUDA 12.9 drivers". The R580 and R595 data-centre drivers that the GPU Operator deploys are newer than that floor.

Worker node sizes for OpenShift AI

Our three reference configurations map onto OpenShift worker roles. They follow NVIDIA's guide for NVIDIA-Certified Systems: two sockets, at least six cores per GPU and system memory of at least twice the GPU memory.

Our reference configurations on /ai-servers/, each with two AMD EPYC processors: 384 GB, 768 GB and 2.3 TB of DDR5 ECC, the H200 NVL node with 400G networking; usable memory by our users-per-card rule (0.9 × driver-visible memory − 3 GiB per card); wall power is our estimate (cards at 600 W, two EPYC 9135, 9355 or 9555 respectively, and 15 per cent for the rest), confirmed by a measurement on the delivered system.

As examples by our users-per-card rule from our sibling pages: four RTX PRO 6000 Server Edition cards hold Llama 3.3 70B in FP8 for 156 concurrent conversations of 8K as two tensor-parallel pairs with an FP8 KV cache, and four H200 NVL hold 340 as one copy. A cluster that serves one large model and many small ones often combines both: an RTX PRO 6000 Server Edition or H200 NVL node for the main model and an L4 or MIG-split node for embeddings, rerankers and notebooks. The figures per layout are in our pages on the four-card RTX PRO 6000 server and the eight-card H200 NVL server.

At about 3.4 kW, the four-card node exceeds the 2.94 kW we plan per 230 V 16 A feed, and the eight-card node needs a three-phase position; our guide to GPU server power on 16 A, 32 A and three-phase works this out. OpenShift adds nothing to the power budget, but cluster designs with three or more GPU workers for availability multiply it.

What we supply

You can buy these worker nodes from us: our team sizes the configuration to your workload and budget and replies within one business day. We build AI servers to order with RTX PRO 6000 or RTX PRO 4500 Server Edition, H200 NVL, L40S or L4 cards, assembled and tested before delivery, on one EU contract and invoice with manufacturer warranty on every component, and deliver them across the EU. NVIDIA AI Enterprise licences per GPU can come on the same invoice. Send us your OpenShift version, node count and the models you plan to serve, and we reply with a node configuration. If you also want help with models, retrieval and serving, that is our Private AI/ML service.

FAQ