GPUs in VMware vSphere: DirectPath I/O passthrough or NVIDIA vGPU, what each needs and what each allows

Three ways to put a GPU into a VM

DirectPath I/O passes a whole physical GPU through to one VM. Dynamic DirectPath I/O, from ESXi 7.0, is the same passthrough with a looser binding: the VM names acceptable vendor and device combinations, or a hardware label, and ESXi picks a device “based on what is available at the time of VM power-on”. With NVIDIA vGPU, the vGPU Manager in the hypervisor gives each VM a profile with a fixed amount of GPU memory, time-sliced on the card’s engines or backed by a MIG instance. A card does one or the other: it can host vGPUs or be passed through “but cannot do both at the same time”, and on ESXi the switch from passthrough to vGPU needs a host reboot.

Broadcom KB 312208; NVIDIA vGPU 20 user guide and vSphere release notes; NVIDIA AI Enterprise 8.2 support matrix (passthrough driver: vGPU guest or data-centre), September 2026.

What each mode allows

Passthrough gives the guest the full card and costs the VM its mobility. Broadcom lists suspend and resume, fault tolerance, high availability, snapshots and hot-add of virtual devices as unavailable for DirectPath I/O VMs, and DRS only in a limited form: the VM can sit in a cluster but cannot migrate. Snapshots succeed once the VM is powered off, which matters for backup products that work through snapshots, and the VM needs a memory reservation for all of its configured memory. Dynamic DirectPath I/O changes placement, not mobility: DRS in vSphere 7 can use Assignable Hardware “to place your VM on a suitable host server”, VMware wrote in 2020.

Broadcom vSphere 9.0 DirectPath I/O page, KB 312208, KB 428701, vSphere 8.0 graphics documentation; VMware blog, April 2020; NVIDIA vGPU 20 release notes and known issues for vSphere.

Monitoring differs too: NVIDIA states that DCGM “is not supported in vGPU environments”.

Supported cards and versions, September 2026

NVIDIA’s current vGPU release, 20.2 of August 2026, sits on the R595 production branch, supported until March 2027. The long-term vGPU 19 branch, on R580, is supported until July 2028 according to NVIDIA’s vGPU release table, while NVIDIA’s data-centre driver table lists June 2028 as the end of life of the R580 driver branch. Release 20.2 pairs vGPU Manager 595.91.04 with guest drivers 595.91.07 (Linux) and 596.86 (Windows) and supports VCF 9.1, VCF 9.0 with its maintenance releases, and ESXi 8.0 Update 3 P06 (released as 8.0 Update 3g, build 24859861) or a later 8.0 update. NVIDIA AI Enterprise 8.2 uses the same drivers, lists ESXi 8.0 and 9.0 and later, and supports its GPUs “with servers listed on the NVIDIA-Certified Systems”.

NVIDIA vGPU 20 support matrix for vSphere (22 September 2026) and supported-GPU list; vGPU 19 release notes for vSphere; NVIDIA AI Enterprise 8.2 support matrix and H200 vGPU types; H200 NVL product brief.

The H200 NVL is not in NVIDIA’s vGPU product: since release 16.0 that product leaves GPUs with only C-series profiles to NVIDIA AI Enterprise. Among RTX PRO Blackwell cards we found only the Server Editions in NVIDIA AI Enterprise 8.2’s GPU list.

Preparing hosts and VMs

NVIDIA’s vGPU guide asks for VT-d or IOMMU, SR-IOV and Alternative Routing ID Interpretation (ARI) in the BIOS for GPUs from the Ampere generation on; NVIDIA AI Enterprise recommends memory-mapped I/O above 4 GB where the BIOS offers it. Passthrough needs the IOMMU and, per Broadcom’s KB 312208, ACS in the PCIe root ports and in the switch downstream ports above the device.

The vGPU Manager is an ESXi driver, which Broadcom’s validated design for private AI adds to the cluster’s vSphere Lifecycle Manager image; nvidia-smi on the host then lists the GPUs. Under Configure, Hardware, Graphics, each GPU’s device type must then be Shared Direct, followed by a host restart. Left at Shared, VMware’s vSGA, vGPU VMs fail to start with an error about insufficient graphics resources: “NVIDIA vGPU software does not support VMware vSGA”. The L40S and RTX PRO 6000 Server Edition must run vGPU in display-off mode, their factory setting unless it was changed.

For passthrough, select the physical GPU, not its virtual functions, under Configure, Hardware, PCI Devices, toggle passthrough, reboot, and add the GPU to the VM as a PCI device with all memory reserved. Broadcom’s procedure for moving vGPU hosts to passthrough disables MIG (nvidia-smi -i 0 -mig 0 for GPU 0), removes the vGPU host driver with esxcli software vib remove -n and the driver name, and reboots.

For passthrough of “GPUs with large BAR memory settings”, NVIDIA requires a 64-bit guest and 64-bit MMIO for the VM, plus EFI boot once the total BAR1 memory exceeds 256 MB; Broadcom’s parameters are pciPassthru.use64bitMMIO set to TRUE and pciPassthru.64bitMMIOSizeGB, “a power-of-two number of GB”, which NVIDIA sizes as each GPU’s BAR1 total from nvidia-smi -q times the GPUs in the VM (4 × 128 GiB gives 512). A vGPU VM with 32 GB or more of memory needs the same settings if its GPU needs 64 GB or more of MMIO, with the window raised to the GPU’s figure: 512 GB for H200 cards and 64 GB for the L4 in NVIDIA AI Enterprise’s table, which gives none for the L40S or RTX PRO 6000 Server Edition. And bridged H200 NVL cards go to one VM together: in passthrough, “all GPUs connected to each other through NVLink must be assigned to the same VM”, or the VM hits XID 74 at boot.

MIG-backed vGPU on ESXi

MIG is enabled per GPU on the host, with the vGPU Manager installed and nothing else using the GPU: nvidia-smi -i 0 -mig 1 for GPU 0. ESXi needs a host reboot for this only with Ampere GPUs, and vSphere “creates the GPU instances automatically”; non-default compute instances inside a MIG-backed vGPU are not supported on vSphere. On vSphere, MIG-backed graphics vGPUs that each fill a GPU instance are supported since vGPU 19.0, while time-sliced vGPUs inside a GPU instance need VCF 9.1 (not VCF 9.0 or ESXi 8) with vGPU 20.0 or 19.4 onwards, and several MIG-backed vGPUs per VM need VCF 9.1 with 20.2 or 19.6 onwards, in NVIDIA AI Enterprise as well. On the RTX PRO 6000 Server Edition, time slicing inside MIG gives up to four MIG slices with one to three 8 GB compute vGPUs each. MIG-backed vGPUs cannot live-migrate between different MIG profiles, and on H200 cards Windows guests are limited to time-sliced vGPUs. Profile counts per card are in our MIG and vGPU guide.

vMotion and DRS with vGPU

vMotion of vGPU VMs is enabled in vCenter under Configure, Settings, Advanced Settings: vgpu.hotmigrate.enabled set to true, added if missing. The destination needs, per NVIDIA, a GPU of the same type, the same ECC setting and the same GPU topology, NVLink widths included; unified memory, CUDA debuggers and profilers disable migration, and H200, H800 and H100 boards cannot migrate to one another. Direction matters: resuming on a host from an older main vGPU branch fails, and a 20.1 VM moved to a 20.0 host shuts down, so upgrade hosts first and move VMs only to the same or a newer release.

The VM is inaccessible for part of the move, and Broadcom notes that this stun time “can vary depending on the amount of GPU memory that is currently being consumed by the VM”. From vSphere 8.0 Update 2 a stun time limit can be set per VM, vCenter estimates the stun time (in 8.0 Update 2 for C-series and Q-series profiles, with a vMotion network on each host), and DRS migrates a vGPU VM automatically when its estimated stun time is below its own limit and the cluster options PassthroughDrsAutomation and LBMaxVmotionPerHost are set to 1 and VmDevicesStunTimeTolerated, 100 seconds by default, exceeds the largest estimated stun time in the cluster.

Licences

NVIDIA’s vGPU product for vSphere has no compute profiles: “C-series vGPU types are not available. Instead, NVIDIA vGPU for Compute is supported with NVIDIA AI Enterprise.” That licence is per GPU and covers up to 16 vGPUs on it, or one vGPU using the whole card. Each VM checks a licence out at boot from the NVIDIA License System, in NVIDIA’s cloud or on your own network; without one it runs at full capability for 20 minutes, then drops to an idle level of compute. Graphics profiles take RTX vWS (Q-series), vPC (B-series) or vApps (A-series), counted per concurrent user; RTX vWS and vApps also cover GPU passthrough. A compute VM in passthrough with the data-centre driver needs no licence server, as the License System “is only required” for vGPU for Compute drivers, though NVIDIA AI Enterprise software in it still needs its per-GPU licence. Counting rules and included subscriptions, such as the H200 NVL’s five years, are in our licensing guide; which vSphere edition includes vGPU is covered in our renewal guide.

Designing the GPU cluster

Passthrough fits VMs that need whole cards: a model spread over several GPUs, bridged H200 NVL cards, long fine-tuning runs. vGPU fits many smaller VMs that share cards and should stay movable: virtual workstations, development machines, small inference services.

The rules above favour a separate cluster for GPU hosts. A vGPU VM moves only to a host with the same GPU type; the vGPU DRS settings are cluster options, so LBMaxVmotionPerHost at 1 applies to every VM in the cluster; and with image-based lifecycle management the vGPU Manager is part of the cluster image. Broadcom’s HA admission control is calculated on CPU and memory, and we found nothing in it about GPUs, so keep enough free GPUs of each type to absorb a host in maintenance or a failed one. Passthrough VMs, which can neither vMotion nor suspend, have to be shut down before their host enters maintenance mode. How many cards a host takes depends on lanes, power and airflow, as our server sizing guide sets out.

GPU hosts touch editions, versions and maintenance windows at once, the ground our engineering partner Vixen.UNO covers in VMware optimisation: an audit of the estate and its licences, editions matched to real workloads within Broadcom’s licensing logic, and vSphere and VCF updates in agreed maintenance windows with a rollback plan.

What we supply

Of the cards NVIDIA supports on vSphere, Eurokommerz supplies the RTX PRO 6000 Server Edition, L40S, L4 and H200 NVL, with manufacturer warranty on an EU contract and invoice, as cards or in AI servers built to order. NVIDIA AI Enterprise and vGPU licences go on the same quote and invoice as the hardware, as our licensing page explains. Under the same contract, our engineering partner Vixen.UNO delivers VMware optimisation: the estate and licence audit, editions matched to real workloads, vSphere and VCF modernisation in agreed maintenance windows with a rollback plan, and support under an agreed SLA.

FAQ