Buying a GPU server for LLMs in Europe: 2, 4 or 8 GPUs, what each runs and how to choose

GPU servers for LLMs with 2, 4 or 8 GPUs

You can buy a GPU server for large language models from us with two, four or eight GPUs, built to order and delivered anywhere in the EU. For most companies the card is the NVIDIA RTX PRO 6000 Blackwell Server Edition with 96 GB per card: two cards run a 70B model for a pilot or one department, four run several models in production, and eight hold the largest open models. For fine-tuning and training, or where every user needs a fast stream from a 70B model, the NVIDIA H200 NVL with NVLink bridges is the stronger choice.

This page is the overview of our GPU server range for LLMs. It lists our reference configurations and their variants, shows what each runs by one sizing rule, and links to a buy page and a technical page for every configuration. Our guide on where to buy an AI server in Europe compares buying channels and warranty, and buying an AI server: what to specify goes through the specification line by line; here the starting point is the card count.

Three reference configurations and their variants

Our AI servers start from three builds. Memory and processor cores follow NVIDIA's configuration guide for NVIDIA-Certified Systems, updated on 30 September 2026: at least two sockets, at least six physical cores per GPU and system memory of at least twice the total GPU memory. More memory or cores are available on request.

Our AI servers page; memory and cores by NVIDIA's NVIDIA-Certified Systems configuration guide, read on 24 October 2026.

Between and beyond these three we build the variants below on the same platform rules.

NVIDIA product pages for the RTX PRO 4500 Blackwell Server Edition and the H200 NVL; host minimums by NVIDIA's configuration guide (six cores per GPU, memory twice the GPU memory).

The RTX PRO 4500 Blackwell Server Edition is a passive single-slot card with 32 GB of GDDR7 at 800 GB/s and 165 W, and NVIDIA lists up to two MIG instances of 16 GB per card. Supermicro lists eight of them in its 3U to 5U RTX PRO servers. The card suits a server that runs many small models side by side, such as eight copies of a 14B model in FP8, or virtual desktops, since it is on NVIDIA's vGPU list from release 20.0. The H200 NVL has 141 GB of HBM3e at 4.8 TB/s, up to 600 W, and NVIDIA's bridges join two or four cards at 900 GB/s per GPU, against 128 GB/s for PCIe Gen5. The RTX PRO 6000 Server Edition has no NVLink, so its cards exchange data over PCIe.

For a rack, we recommend the Server Edition of the RTX PRO 6000 for continuous operation. The RTX PRO 6000 Max-Q is the workstation card, a 300 W blower card that NVIDIA supports up to four per workstation; in a rack server it belongs only in a chassis whose maker lists it, and many makers do not list it for their GPU servers.

What each configuration runs

Our users-per-card rule takes 90 per cent of the memory the driver reports, subtracts 3 GiB per card for the engine and subtracts the weights; what is left holds KV cache, the memory each conversation keeps on the GPU. The driver reports 95.6 GiB on a 96 GB RTX PRO 6000 and 140.4 GiB on a 141 GB H200 NVL, so one card holds 83.0 or 123.4 GiB. The table counts conversations held at their full context length at the same moment.

Our estimates by the users-per-card rule. Llama 3.3 70B: NVIDIA's FP8 checkpoint, 67.7 GiB, with an FP8 KV cache of 1.25 GiB per 8,192-token conversation. gpt-oss-120b: 60.8 GiB, 16-bit cache of 1.125 GiB per 32,768-token conversation. The RTX PRO 4500 Server Edition is scaled from 95.6 GiB per 96 GB to 25.7 GiB per card.

As a worked example, one TP=2 pair of RTX PRO 6000 Server Edition cards holds 2 × 83.04 = 166.1 GiB; less 67.7 GiB of Llama 3.3 70B weights that leaves 98.4 GiB, and 98.4 ÷ 1.25 gives 78 conversations of 8K. The same model on one card alone leaves 15.3 GiB, 12 conversations, which is why a 70B model on the RTX PRO 6000 Server Edition usually runs as a pair. On one H200 NVL it leaves 55.7 GiB and 44 conversations, so it runs one copy per card.

For larger models, the model size sets the card count. Eight Server Edition cards hold Llama 4 Maverick in FP8, about 417 GB. A checkpoint of about 690 GB such as DeepSeek-V3.2 would leave about 22 GiB of cache on eight Server Edition cards by our arithmetic, too little for real use, and belongs on eight H200 NVL. Our LLM hardware requirements by model lists the sizes; each configuration page linked below has its own model table.

These are requests in flight, which is far fewer than the number of staff who use a service. Our guide to a private ChatGPT server by company size converts headcount into requests in flight, and how many users one RTX PRO 6000 can serve derives the rule. We supply every configuration in the tables; send us the models and the requests you expect at the busiest hour, and our team replies with a configuration and the figures for your model.

Choosing in five questions

Five answers decide which of these servers fits, in this order.

1. Which model, at which precision? The weights in FP8 take about one byte per parameter, in NVFP4 about 0.56 bytes, in BF16 two. A model that fits one card with room for its cache runs as one copy per card; a larger one is split over two, four or eight cards. Our article on GPU server sizing for a 70B model works through one case.
2. How many requests are in flight at the peak? The table above gives the capacity per layout. If the peak is below the capacity of one pair, two cards are enough; several models side by side, such as an assistant, a coding model and speech-to-text, point to four cards.
3. Must the service survive the loss of a server? One eight-card server is one point of failure. Two four-card servers, each able to carry the whole peak, keep the service running during a fault or an update, as our comparison of one 8-GPU server or two 4-GPU servers explains.
4. Inference only, or fine-tuning and training as well? Training exchanges gradients between cards at every step, which NVLink carries at 900 GB/s per GPU on the H200 NVL; the RTX PRO cards use PCIe. Our fine-tuning server guide compares 2, 4 and 8 GPUs for 8B to 70B models.
5. What does the rack position deliver? A 230 V 16 A feed per side carries a two-card server or four cards capped at 450 W; four cards at 600 W need 32 A or three-phase, and eight cards three-phase. The section on power below has the figures.

If virtual machines are planned, add a sixth check: the RTX PRO 6000 Server Edition is on NVIDIA's vGPU list from release 19.0 and the RTX PRO 4500 Server Edition from 20.0, while the H200 NVL runs compute vGPU profiles only through NVIDIA AI Enterprise. The choice between the two card families is set out in our RTX PRO 6000 vs H200 NVL comparison and, by card count, in RTX PRO 6000 Server Edition: 2, 4 or 8 GPUs compared.

Published MLPerf® results by card count

MLPerf® Inference v6.0 has results on Llama 2 70B for two, four and eight RTX PRO 6000 Server Edition cards and for eight H200 NVL, which show how throughput scales with the card count.

MLPerf® Inference: Datacenter v6.0, closed division, available category, Llama 2 70B (llama2-70b-99), offline scenario, entries 6.0-0052, 6.0-0054, 6.0-0047 and 6.0-0021, retrieved from mlcommons.org on 24 September 2026. Results verified by MLCommons Association. Per card is our division of the system result, not an MLPerf metric. The MLPerf name and logo are registered and unregistered trademarks of MLCommons Association in the United States and other countries. All rights reserved. Unauthorized use strictly prohibited. See www.mlcommons.org for more information.

The offline scenario measures throughput with all requests available at once, the best case for a batch job. In the newer round, MLPerf® Inference: Datacenter v6.1, closed division, HPE's ProLiant Compute DL380a Gen12 with eight RTX PRO 6000 Server Edition reached 14,581.7 tokens per second on gpt-oss-120b in the server scenario, where requests arrive under a latency limit, entry 6.1-0050, retrieved from mlcommons.org on 24 October 2026, result verified by MLCommons Association. Our RTX PRO 6000 benchmarks and H200 NVL benchmarks add the other scenarios.

Power and room for each card count

By our estimate, a server draws its cards' board power, two processors and 15 per cent on top for memory, drives, network cards, fans and conversion losses. A measurement on the delivered system confirms the figure.

Our estimate: board power from NVIDIA (RTX PRO 6000 Server Edition and H200 NVL up to 600 W, RTX PRO 4500 Server Edition 165 W); two processors at AMD's default TDP: EPYC 9135 at 200 W each with two cards, EPYC 9355 at 280 W with four cards and with eight RTX PRO 4500 Server Edition, 64-core EPYC 9555 at 360 W with eight 600 W cards; plus 15 per cent. Feeds planned at 80 per cent: 2.94 kW at 230 V 16 A, 5.89 kW at 32 A, 8.87 kW at 400 V 16 A three-phase.

Dell's AI Factory datasheet gives about 8 kW at full load for its eight-card RTX PRO 6000 Server Edition nodes, more than our estimate, so we plan an eight-card position on a three-phase feed from the start. With redundant power supplies on A and B feeds, each side has to carry the whole server. Every kilowatt becomes heat: about 6.3 kW needs roughly 1,000 CFM of air at an 11 °C rise by the 160 CFM per kW in our guide to GPU rack power and cooling. Any of these servers belongs in a server room or data centre, and our checklist for preparing a server room for a GPU server lists what to check.

Software on delivery

On request the server arrives with the operating system, the NVIDIA driver, CUDA and a container runtime installed, so your team can start a serving engine such as vLLM or SGLang on the first day. The RTX PRO 6000 Server Edition needs Linux driver R575 or later and CUDA 12.9 or later on x86, by NVIDIA's product brief SP-12355-001. As of October 2026, R580 is NVIDIA's long-term branch with support to June 2028, and NVIDIA's security bulletin of 30 September 2026 names 580.178.04 and 595.91.07 as fixed drivers; our article on NVIDIA driver branches and CUDA versions explains the choice.

Open-source engines on bare metal need no NVIDIA licence. NVIDIA AI Enterprise, licensed per GPU, is needed for NIM and other NVIDIA AI Enterprise software and for compute vGPU profiles, and it comes on the same invoice as the hardware. Each H200 NVL includes a five-year NVIDIA AI Enterprise subscription; by NVIDIA's licensing guide of 2 September 2026 it starts 90 days after the card ships to the server maker. Models, RAG and the platform on top are our Private AI/ML service.

How ordering works with us in the EU

The steps are the same for one server or several.

1. You describe the workload: models, users, data, rack position and timeline. Sizing is free of charge.
2. We propose a configuration and check that cards, chassis, power supplies and cooling fit your rack and feed.
3. You receive a written quote within one business day, with price, lead time and warranty terms per component. Quantities and lead time are confirmed in writing with the quote.
4. You order on one EU contract with an EU invoice from Eurokommerz in Vienna; NVIDIA licences are lines on the same invoice.
5. The server is assembled and tested before delivery.
6. We deliver anywhere in the EU, with commissioning on site on request; other destinations on request through our logistics partner.
7. Every component carries the manufacturer warranty, handled through us; DOA units are replaced, and the assembly is covered by us.

We also build on your own chassis or with parts you already own, after a compatibility check of the platform, power and cooling. To compare quotes from several suppliers line by line, use our guide to GPU server quote comparison.

What to send us for a quote

A short description is enough; these points let us quote an exact configuration in the first reply.

• The models you plan to run, with precision and the context length you will declare
• Peak requests in flight, or the number of users and how they work
• Other workloads on the same server: embedding and reranking for RAG, speech-to-text, fine-tuning, virtual machines or MIG
• Whether the service must survive the loss of one server
• The rack position: free height units, feed type, A/B feeds, PDU outlets and inlet temperature
• The software platform (bare metal, Kubernetes or vSphere), delivery site and invoicing entity

Describe these points in the form below, and our team replies with a configuration and a written quote within one business day.

Buy pages and technical pages for each configuration

Each server configuration has a buy page with the parts list, variants and ordering details, and a technical page with models, users and power.

For a team that wants the GPUs under the desk or in a lab, the workstation pages cover two to four cards.

What we supply

You can buy this system from us: our team sizes the configuration to your workload and budget and replies within one business day. We build GPU servers to order with two to eight RTX PRO 6000 Server Edition, RTX PRO 4500 Server Edition or H200 NVL cards, assembled and tested before delivery, with manufacturer warranty on every component handled through us, and supply the GPUs on their own for servers you already run. NVIDIA AI Enterprise and vGPU licences come on the same EU contract and invoice. Our engineering partner Vixen.UNO deploys models, RAG and the platform on top as our Private AI/ML service; tell us which models and how many users you plan for, and we size server and platform together.

FAQ