Replacing A100 PCIe cards: RTX PRO 6000 Server Edition or H200 NVL, and what changes for power, cooling, software and MIG

What you are replacing

The A100 PCIe came in two versions: 40 GB of HBM2 at 1,555 GB/s and 250 W, and 80 GB of HBM2e at 1,935 GB/s and 300 W. Both are passive, dual-slot, full-length PCIe 4.0 x16 cards that pair through NVLink bridges at 600 GB/s; the 80 GB card also came in a single-slot liquid-cooled version. NVIDIA still supports them in software: its vGPU lifecycle page, updated on 4 August 2026, lists the A100 PCIe and A100 PCIe 80GB among the GPUs that “have not reached the end of the full support phase”, and CUDA 13.0 dropped Maxwell, Pascal and Volta, not Ampere. The reasons to replace are memory, number formats and bandwidth. The two PCIe cards that take over the role are the H200 NVL and the RTX PRO 6000 Blackwell Server Edition, and they differ from each other as much as from the A100.

The four cards side by side

NVIDIA A100 datasheet (June 2021) and product briefs PB-10137 (2020) and PB-10577 (2022); H200 product page and H200 NVL product brief PB-12128 (April 2025); RTX PRO 6000 Server Edition product page, datasheet (December 2025), product brief SP-12355 (June 2025) and RTX PRO Blackwell architecture whitepaper; MIG user guide (September 2026). CC is CUDA compute capability, SE the Server Edition. Dense H200 figures are NVIDIA’s sparse figures halved. The RTX PRO 6000 BF16 and FP64 figures are our arithmetic from the whitepaper, dense BF16 at four times and FP64 at 1/64 of the Server Edition’s 120 TFLOPS of FP32; NVIDIA’s 1 PFLOP BF16 headline for the card states no sparsity basis.

Read the power, connector and host-link rows first: they decide whether a given server can take either card at all. The memory and format rows decide what the card will run, and the FP64, NVLink and MIG rows decide which A100 jobs can move to only one of the two.

Formats: what FP8 and FP4 change for serving

The A100’s datasheet lists Tensor Core rates for FP64, TF32, BF16, FP16 and INT8, and none for FP8. vLLM’s documentation states that FP8 computation needs compute capability 8.9 or higher and that FP8 models run on older GPUs “as weight-only W8A16, utilizing FP8 Marlin”: on an A100 an FP8 checkpoint halves the weights, but the arithmetic stays 16-bit. TensorRT-LLM’s support matrix of 21 September 2026 gives Ampere INT4 AWQ and GPTQ with 16-bit activations and an FP8 KV cache, and no FP8 or FP4 arithmetic. Hopper adds FP8 with per-tensor, block and row scaling, plus INT4 with FP8 activations. The RTX PRO 6000 generation adds NVFP4 and MXFP4, while the same matrix lists only per-tensor FP8 and no AWQ or GPTQ for it.

The memory side for Llama 3.3 70B, with a planning rule of 90 per cent of the memory the driver reports minus the weights: in FP8 the model is 68 GiB. On an A100 80 GB that leaves about 4 GiB, and the model runs weight-only; in BF16, at about 131 GiB, it needs two A100 80 GB cards. On the H200 NVL, with 140.4 GiB visible, it leaves 58.4 GiB, cache for about 46 conversations of 8,192 tokens in FP8. On the RTX PRO 6000, with 95.6 GiB visible, it leaves 18 GiB, about 14 conversations, or 46 GiB and about 36 conversations with the 40 GiB NVFP4 checkpoint.

For a single user, generation follows bandwidth, and here the RTX PRO 6000 is no step up: 1,597 GB/s against the A100 80 GB’s 1,935. Its gains are FP4 weights, which cut the bytes read per token by about 40 per cent against FP8 (40 against 68 GiB for Llama 3.3 70B), native FP8 and FP4 arithmetic for the compute-bound prompt phase, and 16 GB more memory. The H200 NVL is the bandwidth upgrade, at 4,800 GB/s; our inference comparison sets the two against each other.

Double precision separates them further. The H200 NVL delivers 30 TFLOPS of FP64 and 60 on its Tensor Cores, three times the A100’s 9.7 and 19.5. NVIDIA’s RTX PRO Blackwell whitepaper says the GB202 chip has two FP64 cores per SM, runs FP64 at 1/64 of its FP32 rate and includes them “to ensure any programs with FP 64 code operate correctly”. That puts the Server Edition at about 1.9 TFLOPS by our arithmetic, a fifth of an A100. Simulation codes that ran in double precision on A100s belong on the H200 NVL.

MIG and vGPU

Both A100 versions and the H200 NVL split into up to seven instances, 1g.5gb or 1g.10gb on the A100 and 1g.18gb on the H200 NVL, which NVIDIA lists as 16.5 GB each; the RTX PRO 6000 splits into four of 24 GB, with +gfx profiles that run graphics inside an instance. Two details of operation change with the generation. On Ampere, enabling MIG makes the driver attempt a GPU reset, and MIG mode is stored in the InfoROM and survives a reboot. From Hopper on, no reset is needed, but MIG mode lasts only while the driver is loaded. On every card the instances themselves have to be recreated after a reboot, for which NVIDIA points to its mig-parted tool run as a systemd service. Boot automation written for A100s needs a MIG-mode step on the new cards.

For virtual machines the difference is what the slices may do. The A100 and the H200 NVL take compute (C-series) vGPU profiles only, licensed through NVIDIA AI Enterprise: from A100-1-5C on the 40 GB card and A100D-1-10C on the 80 GB card upwards, and seven H200-1-18C instances or up to 32 time-sliced 4 GB profiles on the H200 NVL. The RTX PRO 6000 Server Edition has been on NVIDIA’s vGPU list since release 19.0, with virtual workstation and desktop profiles as well as compute ones. Of the two replacements, the H200 NVL comes with a five-year NVIDIA AI Enterprise subscription; we found no such inclusion for the RTX PRO 6000 Server Edition.

Power, cooling and the cable

The power per slot doubles or more: 250 or 300 W becomes up to 600 W, and four cards go from 1.0 or 1.2 kW to 2.4 kW before processors and fans. The connector changes as well. The A100 PCIe takes one CPU 8-pin auxiliary connector; both replacements take a PCIe 16-pin connector whose sense pins tell the card what the supply can deliver. The product briefs of both cards say that if the level signalled “is less than the default power cap of the NVIDIA card, the card will not boot.” The H200 NVL supports only the 451 to 600 W level. The RTX PRO 6000 Server Edition also accepts 301 to 450 W and then caps itself at 450 W; Lenovo, for example, lists four of these cards in an SR650a V4 when capped to 450 W, against two at 600 W.

Airflow and the slot follow. The RTX PRO 6000 brief asks for at least 43 CFM of ducted air through the heat sink at a 25 °C inlet, 65 CFM at 35 °C and 125 CFM at 45 °C; the H200 NVL brief publishes no rate. Both briefs accept a Gen5 x16, Gen5 x8 or Gen4 x16 host link, so a PCIe 4.0 server does not rule either card out, although a Gen4 x16 link halves the bandwidth to the host.

None of this is settled by the card alone. It is settled by the server maker’s GPU support list for your exact machine type. Dell’s PowerEdge GPU matrix of August 2026 lists both cards only on seventeenth-generation servers and does not list the A100 at all. Our H200 NVL checklist goes through the rest: cable part numbers per slot, fan kits, air ducts, drive backplanes and power supplies.

Drivers, CUDA and containers

The software floor moves up. NVIDIA’s H200 NVL brief requires driver R565 TRD1 or later and CUDA 12.7, a version NVIDIA’s toolkit archive does not list, so in practice 12.8 or later; the RTX PRO 6000 Server Edition brief requires R575 or later and, on x86, CUDA 12.9 or later. In NVIDIA’s data-centre driver release notes, the R570 branch lists the A100 and the H200 NVL but not the Server Edition, which first appears in R575. The R580 release of 3 August 2026 lists the A100, the A100 80GB PCIe, the H200 NVL and the RTX PRO 6000 Server Edition, as does the R595 release of the same day, so one driver can run a server in transition. In NVIDIA’s table of supported branches of 9 September 2026, R580 is a long term support branch, supported to June 2028, and R595 the production branch, supported to March 2027. The release notes of R615, a new feature branch not listed in that table, also name all three cards. Images must target the new chips too: CUDA 12.8 was the first toolkit with compiler support for Blackwell’s SM_120, and PyTorch added Blackwell support with CUDA 12.8 builds in version 2.7. A container built on CUDA 11 for the A100 falls below the CUDA versions both briefs require, and it carries no compiled code for Blackwell. Our driver and CUDA guide covers branch choice and container compatibility.

Which card takes which A100 job

Double-precision simulation: the H200 NVL. The RTX PRO 6000 is not an FP64 card.

Bridged pairs running one model across two cards: the H200 NVL, whose bridge joins two or four cards at 900 GB/s per GPU. It uses one bridge per card where the A100 pair used three, so the old set does not carry over, and the RTX PRO 6000 has no NVLink at all.

Interactive serving with long contexts: the H200 NVL, with 141 GB at 4,800 GB/s.

Throughput serving of models that fit 96 GB, next to graphics, video or virtual desktops: the RTX PRO 6000 Server Edition, the card NVIDIA also sets against the L40S.

Isolated tenants: seven per card on the H200 NVL; four on the RTX PRO 6000, with graphics inside the slices.

A server that cannot deliver 600 W, the airflow or the cable to a slot: neither card at full power. The RTX PRO 6000 at 450 W is the fallback where the maker lists it; otherwise the server is what needs replacing.

What we supply

Eurokommerz supplies the H200 NVL and the RTX PRO 6000 EU-wide with manufacturer warranty, as cards or in servers configured with the power supplies, cables and airflow they need. We do not supply the A100; send us the server model and the workload, and we will check which replacement its maker supports.

FAQ