GPUs in Kubernetes: what the GPU Operator installs, and four ways to share a card

What the GPU Operator installs

NVIDIA’s GPU Operator uses the Kubernetes operator framework “to automate the management of all NVIDIA software components needed to provision GPU”. This guide follows release v26.7.1. Release v26.7.2, in NVIDIA’s release notes updated 14 October 2026, lists only the DRA driver, now v0.5.1, among its updated components and keeps 595.91.07 as the default driver. As of 23 October 2026, install 26.7.2 or later: Operator releases up to 26.3.1 are affected by CVE-2026-24260 (NVIDIA Container Toolkit up to 1.19.0). On GPU nodes the Operator deploys these components, each as a container.

NVIDIA GPU Operator documentation: release notes, installation options and platform support, updated 23 September 2026; NVIDIA DCGM Exporter documentation, May 2026.

Node Feature Discovery and validator pods complete the set, and toolkit.enabled=false keeps a container runtime that is already configured for NVIDIA. Our monitoring guide covers what to watch in the DCGM metrics. Check NVIDIA’s support list for the Operator first: it names data-centre cards such as the H200 NVL, L40S and L4 and, among the Blackwell PCIe cards, the RTX PRO 6000 and RTX PRO 4500 Server Editions. We found no entry for the RTX PRO 6000 Workstation or Max-Q editions or for the RTX PRO 5000. For the RTX PRO 6000 Server Edition the list requires driver 575.57.08 or later and notes that MIG is not supported on the 575.57.08 driver release itself.

From that list we supply the RTX PRO 6000 and RTX PRO 4500 Server Editions, the H200 NVL, the L40S and the L4, as cards or in AI servers built to order. Tell us the servers your cluster runs on, and we say which of these cards fit them.

Four ways to share a GPU in Kubernetes

A pod asks for a whole number of GPU resources; the device plugin decides what each unit is: a physical card, a MIG instance or a replica of a shared card. The main difference between the four methods is how far one pod can affect its neighbours.

NVIDIA MIG user guide (4 November 2026), GPU Operator documentation (23 September 2026), NVIDIA k8s-device-plugin README and CUDA MPS documentation, read September and October 2026.

Whole GPUs: the default

Without further configuration the device plugin advertises each physical card as one nvidia.com/gpu, and a pod asks for it in its resource limits, for example nvidia.com/gpu: 1. The card then belongs to that container until the pod ends. It is the right unit for training, for models that need most of a card’s memory and for services that keep a GPU busy. The device plugin’s README, which assumes the NVIDIA runtime is the node’s default runtime, adds a warning: if a pod does not request GPUs, the plugin “exposes all the GPUs on the machine inside your container”. Since v25.10.0 the Operator injects GPUs through CDI by default and no longer makes the nvidia runtime class the default handler, so check which set-up your nodes run.

MIG: partitions in hardware

MIG divides a card into GPU instances with their own streaming multiprocessors, memory and memory paths. In NVIDIA’s words, “the on-chip crossbar ports, L2 cache banks, memory controllers, and DRAM address busses are all assigned uniquely to an individual instance”, which gives each instance a defined quality of service “with fault isolation”. Of the cards we supply, NVIDIA’s MIG guide, updated 4 November 2026, lists all three RTX PRO 6000 editions with up to four instances of 24 GB, the RTX PRO 5000 with two, the RTX PRO 4500 with two of 16 GB and the H200 NVL with seven, whose smallest profile is 1g.18gb. For the RTX PRO 4500 the guide names no edition; since its update of 4 November 2026 it gives that card a minimum driver, 575.64, but no minimum vBIOS and no display-mode switch, which it documents only for the RTX PRO 6000 and 5000, whose minimum drivers are 575.51.03 and 575.57.08. We found no mention of MIG in the workstation card’s datasheet or on its product page, so have MIG support for that card confirmed in writing. The guide covers the 48 GB RTX PRO 5000, split into two of 24 GB; for the 72 GB card NVIDIA’s datasheet gives two of 36 GB. NVIDIA’s datasheet of October 2026 gives the RTX PRO 5500 two instances of up to 42 GB; the card is not in the guide yet. The L4, L40S, RTX PRO 4000 and RTX PRO 2000 are not on NVIDIA’s MIG list. Our MIG and vGPU comparison covers the virtual machine side.

The device plugin exposes MIG through one of two strategies, set by the Operator’s mig.strategy value, which defaults to single. With single, MIG instances are advertised under the usual nvidia.com/gpu name, and the resource now stands for the MIG devices on the node instead of the full GPUs. With mixed, every profile becomes its own resource type. Following NVIDIA’s naming pattern, a quarter of an RTX PRO 6000 then appears as nvidia.com/mig-1g.24gb and the smallest H200 NVL instance as nvidia.com/mig-1g.18gb. NVIDIA’s rules for choosing are short: single is for nodes on which MIG is enabled on all GPUs, mixed for nodes on which it is not, or whose layout uses more than one profile. A node that keeps a second NVIDIA GPU out of MIG, such as a display card, therefore needs mixed. Device plugin v0.20.0, which came with Operator v26.7.0 (v26.7.1 ships v0.20.1), fixed its profile matching so that variants with suffixes such as -me, +me.all and +gfx are exposed as separate Kubernetes resources.

MIG Manager applies the layout. A node label, nvidia.com/mig.config, names it, for example all-disabled or a layout with one profile throughout such as NVIDIA’s example all-1g.10gb; since Operator v26.3.0, MIG Manager generates the available layouts for each node from the GPUs it finds, and reports progress in nvidia.com/mig.config.state. Changing the layout is disruptive: NVIDIA states that MIG Manager requires that no user workloads run on the GPUs being configured, and that the node might need to be cordoned and, on some platforms, rebooted. MIG instances cannot be joined into a larger GPU either, and NVIDIA’s MIG guide (updated 4 November 2026) describes the MIG support in NCCL 2.31 and later only as experimental (one rank per MIG instance, CUDA driver 13.0 or later, NCCL_MNNVL_ENABLE=0), as our Nutanix cluster article explains for distributed training.

Three of the workstation cards need one more step that the Operator does not take. On the RTX PRO 6000 Workstation and Max-Q editions and on the RTX PRO 5000, MIG can be enabled only after the display mode has been switched from graphics to compute with NVIDIA’s DisplayModeSelector tool, which turns the display outputs off; NVIDIA supplies the RTX PRO 6000 Server Edition from the factory in display-off mode. We found nothing about this switch in the GPU Operator documentation, so on these cards it is a manual step for each card, described in our MIG runbook for RTX PRO Blackwell.

Time-slicing: more pods, no isolation

Time-slicing lets several pods take turns on one GPU. It is set in a ConfigMap for the device plugin: under sharing.timeSlicing, a resource such as nvidia.com/gpu gets a replicas value, “the number of shared accesses that will be granted for a GPU”, and the ClusterPolicy points to the ConfigMap through devicePlugin.config.name. The field devicePlugin.config.default names the entry for all nodes, and the label nvidia.com/device-plugin.config selects another for a single node; without a default, NVIDIA notes, the configuration does not reach all nodes automatically. With four replicas, each card is advertised as four GPUs. renameByDefault advertises the replicas as nvidia.com/gpu.shared; left at false, the resource name stays the same and the node’s product label gets the suffix -SHARED instead. failRequestsGreaterThanOne rejects any request for more than one replica with an UnexpectedAdmissionError. The Operator does not watch the ConfigMap, so a change applies only after you restart the device plugin pods; running workloads continue, and NVIDIA recommends doing it in a maintenance period.

The trade-off is isolation. NVIDIA’s Operator documentation says that, unlike MIG, “there is no memory or fault-isolation between replicas”, and the device plugin README adds: each workload “has access to the GPU memory and runs in the same fault-domain as of all the others (meaning if one workload crashes, they all do)”. A pod that requests two replicas is not guaranteed twice the compute either. Time-slicing suits development, notebooks and light jobs that tolerate a noisy neighbour. It can also be combined with MIG, to let several pods share each instance.

MPS: fixed shares, one fault domain

CUDA’s Multi-Process Service runs work from several processes on a GPU at the same time through a control daemon. The device plugin supports it with a sharing.mps section that takes replicas like time-slicing, but with limits: the memory each client may use is “limited to an equal fraction of the total device memory”, and the control daemon also caps each client’s share of compute. The README lists the constraints: “As of v0.15.0 of the device plugin, MPS support is considered experimental”, sharing with MPS is not supported on devices with MIG enabled, and only nvidia.com/gpu resources on full GPUs can be shared this way.

NVIDIA’s MPS documentation fills in the fault picture. MPS client processes “have fully isolated GPU address spaces”, but a fatal GPU fault “will be reported to all the clients running on the subset of GPUs in which the fatal fault is contained”, so one faulting process interrupts every pod on that card. NVIDIA describes MPS as useful when “each application process does not generate enough work to saturate the GPU”, which fits many small inference services.

Virtual machines and DRA

The Operator can also give GPUs to KubeVirt virtual machines, as whole cards by passthrough or as vGPUs, chosen node by node with the label nvidia.com/gpu.workload.config; that is a separate set-up from the container sharing described here. The larger change is Dynamic Resource Allocation. Since release v26.7.0 the Operator can manage NVIDIA’s DRA driver, v0.5.1 in v26.7.2; the driver needs Kubernetes v1.34.2 or later and an NVIDIA driver from branch 580 on, and since Kubernetes 1.34 reached end of life on 27 October 2026, plan it on 1.35 or later. Allocation of full GPUs and of existing MIG devices is generally available; creating MIG devices on demand, MPS and custom time-slicing settings are alpha features. Under the Operator, a cluster has either a GPUCluster resource for the DRA driver or a ClusterPolicy for the device plugin, not both.

Choosing a method

Whole GPUs for training and for any service that keeps a card busy on its own.

MIG when several services or teams share a card and each needs guaranteed memory and protection from the others: four instances of 24 GB on an RTX PRO 6000, up to seven on an H200 NVL. Of the cards named here, the H200 NVL and the RTX PRO 6000 Server Edition are on both NVIDIA’s MIG list and the Operator’s support list; the RTX PRO 6000 Workstation and Max-Q editions and the RTX PRO 5000 are on the MIG list only, and need the manual display-mode step. On a server with four RTX PRO 6000 Server Edition, two cards can stay whole for a model served with tensor parallelism over both, while the other two run as eight 24 GB MIG instances for embedding, reranking and small models; since MIG is then not enabled on every GPU of the node, this layout needs the mixed strategy. Our four-card inference node carries four RTX PRO 6000 Server Edition, with the operating system, drivers, CUDA and a container runtime installed on request; name the workloads that should share a card, and we propose a configuration.

Time-slicing for development and light jobs, where a crash that takes the neighbours down is acceptable.

MPS for many small inference processes on one full GPU, with the fault caveat above and not on MIG-enabled cards.

What we supply

We supply these cards across the EU with manufacturer warranty: the RTX PRO 6000 in all three editions, the RTX PRO 5000 and the H200 NVL for MIG, the RTX PRO 4500 workstation card, for which NVIDIA’s guide gives MIG profiles and a minimum driver but names no edition, the RTX PRO 4500 Server Edition, which is on the Operator’s support list, and the L40S and L4 for whole-GPU and time-sliced use. We build AI servers to order with the cards a cluster needs, assembled and tested before delivery, with the operating system, drivers, CUDA and a container runtime installed on request. If you want the Kubernetes platform itself built and supported, that is our Private AI/ML service, built by our engineering partner Vixen.UNO.

FAQ