GPU server for training vision models: detection and segmentation on 2 to 8 GPUs

What a vision training server does

A GPU server for training vision models turns labelled images, such as photos of good and faulty parts from a production line, into detection or segmentation models that run later on small edge devices. For most inspection projects, a server with two or four NVIDIA RTX PRO 6000 Blackwell Server Edition cards trains these models in mixed precision, with 96 GB per card for large batches and high resolutions. Eight RTX PRO 4500 Server Edition cards suit many small experiments in parallel, and eight H200 NVL suit large vision transformers and foundation models.

For a quality manager or plant director, the training server answers a practical question: how quickly a new defect type, a new product variant or a new camera position becomes a working model. Each change means relabelling, retraining and validating. A team that can run four training jobs overnight tries four augmentations or model sizes and picks the best in the morning; a team with one card tries one. The inference side, cameras and line-side hardware are covered in our guides to an AI server for manufacturing and to video analytics.

Model and dataset sizes: where the memory goes

Detection and segmentation models are small compared with language models. Ultralytics' documentation for YOLO11 lists 2.6 million parameters and 6.5 billion FLOPs for YOLO11n and 56.9 million parameters and 195.3 billion FLOPs for YOLO11x, at 640 pixels on COCO. Even the large model takes under 1 GB with its FP32 weights, gradients and two Adam moments: 4 bytes × 4 copies × 56.9 million = 910 MB by our arithmetic. The rest of the card's memory holds activations, which grow with batch size and with the number of pixels: a 1,280-pixel image has four times the pixels of a 640-pixel one.

That is why inspection teams value memory per card. Small defects such as scratches, pores or solder bridges often need higher resolutions, and a larger batch at that resolution fits 96 GB where it does not fit 32 GB. Ultralytics' batch=-1 sizes the batch for about 60 per cent of GPU memory, and a fraction such as batch=0.70 sets another share; on a single GPU the trainer halves the batch and retries after an out-of-memory error in the first epoch, while multi-GPU training stops with the error.

The datasets are modest by data-centre standards. COCO, the usual reference, has 118,287 training and 5,000 validation images in 80 classes, and Ultralytics gives 20.3 GB for the training and validation download. An inspection dataset of a few thousand to tens of thousands of images per product line is smaller still, unless images are high-resolution or video frames are sampled heavily.

Ultralytics documentation for YOLO11 and the COCO dataset, read on 24 October 2026; FLOPs at 640 pixels.

Mixed precision and what the cards deliver

Ultralytics' train mode sets amp=True by default, which "uses FP16"; amp="bf16" selects BF16 on supported CUDA devices, and amp=False trains in FP32. Mixed precision runs the matrix work on the Tensor Cores, so the BF16 and FP16 rates of a card matter more than its FP32 figure.

Published training figures for these cards are rare. The Kempner Institute's computing handbook at Harvard University, updated on 15 July 2026, measured single GPUs with PyTorch 2.8.0 and CUDA 12.8, three runs each. It names the cards RTX PRO 6000 (Blackwell) with 96 GB GDDR7 and H200 with 141 GB, without stating the edition or form factor.

Kempner Institute Computing Handbook, "Which Kempner Partition Should Your Job Run On?", updated 15 July 2026, read on 24 October 2026: 16,384 × 16,384 matrix multiply, 4 GiB device copy, one training step of a 0.94 billion parameter GPT-style model in BF16 autocast; single GPU, means of three runs. Ratios are our division.

The training step is a language model, not a detector, and we found no published training time for a YOLO or segmentation model on the RTX PRO 6000 Server Edition, the H200 NVL or the RTX PRO 4500 Server Edition. Treat the ratios as an indication: per card, an H200-class GPU trained the transformer about 1.9 times as fast. Per server, a four-card RTX PRO 6000 Server Edition machine runs four experiments where two H200 NVL run two. The handbook itself recommends its RTX PRO 6000 partition for single-GPU training and fine-tuning, and the H100 or H200 for multi-GPU model sharding.

Send us your image sizes, dataset size and model family, and our team replies with a configuration that fits your training jobs.

Data loading: processors, RAM and NVMe

A vision training job reads, decodes and augments every image in every epoch, and that work runs on the processors unless it is moved. Ultralytics starts workers=8 data-loading workers "per RANK if Multi-GPU training", so eight cards run 64 workers plus the eight training processes. That is more than the six cores per GPU NVIDIA's configuration guide sets as a minimum; we size the processors for the loaders, for example two 64-core EPYC 9555 in an eight-card server.

Caching helps. Ultralytics' cache=True or cache=ram holds the images in memory and cache=disk on local storage, which "Improves training speed by reducing disk I/O at the cost of increased memory usage." By our arithmetic, COCO's 118,287 training images held as 8-bit RGB arrays at 640 × 640 take up to about 145 GB, and 50,000 inspection images at 1,280 × 1,280 about 246 GB. Our two-card configuration has 384 GB of host memory and the four-card configuration 768 GB, which leaves room for one such cache beside the operating system and the page cache. Ultralytics' train documentation does not say whether each training process of a multi-GPU run keeps its own copy, so check the memory use in a test run before relying on it.

The other option is to move decoding to the GPU. NVIDIA's DALI library, version 2.3.0 on its documentation page, "addresses the problem of the CPU bottleneck by offloading data preprocessing to the GPU", covering "loading, decoding, cropping, resizing, and many other augmentations", and works with PyTorch, TensorFlow, PaddlePaddle and JAX. It needs pipeline code of its own, so teams usually try caching and more processor cores first. For storage, local NVMe drives keep the working copy of the dataset and the checkpoints; our guide to NVMe storage for a GPU server covers drive classes.

One job per card or one job on all cards

Two ways of using several cards make sense for vision models. The first is one experiment per card: different model sizes, augmentations or product lines, each on its own GPU. The second is data-parallel training of one model on several cards, which Ultralytics starts with device=[0,1,2,3]; it "spawns a new trainer instance and executes torch.distributed.run under the hood", that is PyTorch DistributedDataParallel.

Data-parallel training copies the whole model to every card and exchanges only gradients after each step. For YOLO11x that is 56.9 million FP32 gradients, 227.6 MB; with the all-reduce moving about twice that per card, 455 MB at 64 GB/s per direction on PCIe Gen5 take about 7 ms per step, by our arithmetic. That is small next to the computation of a step, so these models scale over PCIe, and the RTX PRO 6000 Server Edition's lack of NVLink costs little here. The H200 NVL's NVLink bridges at 900 GB/s per GPU matter for large vision transformers and foundation models whose states have to be sharded, the case our fine-tuning server guide works through for language models.

MIG gives a third pattern for small jobs. NVIDIA's MIG guide lists up to four instances for every RTX PRO 6000 edition, 1g.24gb each, two of 16 GB for the RTX PRO 4500, which NVIDIA's product page confirms for the Server Edition, and up to seven for the H200 NVL. Each instance runs a single-GPU job with its own memory, which suits hyperparameter sweeps on a small model. The same guide states that NCCL is not supported with MIG, so a data-parallel run needs whole cards; our guide to RTX PRO 6000 MIG profiles has the layouts.

RTX PRO 6000 Server Edition, H200 NVL or RTX PRO 4500 Server Edition

NVIDIA product pages for the RTX PRO 6000 Blackwell Server Edition, the H200 and the RTX PRO 4500 Blackwell Server Edition, as summarised in our research notes of 24 October 2026.

RTX PRO 6000 Server Edition is the general choice for inspection training: 96 GB for high resolutions and large batches, four MIG instances for small jobs, and RT cores for Omniverse and rendering work, which our manufacturing guide covers. H200 NVL fits teams that train large vision transformers or foundation models, where memory bandwidth, 141 GB per card and NVLink bridges for sharding count. RTX PRO 4500 Server Edition fits a team that runs many small detectors in parallel: eight single-slot cards at 165 W give eight training jobs or 16 MIG instances of 16 GB, but 32 GB per card limits resolution and batch size.

Four configurations, from two to eight cards

Reference configurations of our AI servers page and the NVIDIA-Certified Systems configuration guide (30 September 2026). Wall power is our estimate: cards at full board power plus two processors at AMD's default TDP (EPYC 9135 at 200 W each with two cards, EPYC 9355 at 280 W with four RTX PRO 6000 Server Edition and with eight RTX PRO 4500 Server Edition, EPYC 9555 at 360 W with eight H200 NVL), plus 15 per cent for the rest.

Two cards suit one product line or a first project: two experiments overnight, or one data-parallel run on both cards, and the server fits a 230 V 16 A feed planned at 2.94 kW. Four cards suit a central team serving several plants, with four jobs side by side or two MIG-split cards for sweeps; at about 3.4 kW the server needs 32 A single-phase or three-phase power. Eight RTX PRO 4500 Server Edition run eight small jobs at about 2.2 kW, close to the power of the two-card server; our guide to the eight-card RTX PRO 4500 Server Edition server describes it. Eight H200 NVL is our training node, for vision transformers and foundation models, and needs three-phase power or several feeds. Our comparison of eight RTX PRO 6000 or eight RTX PRO 4500 Server Edition weighs the two eight-card options, and our guide to GPU server power plans the feeds.

Training keeps the cards at full load for hours. A server that trains at night can serve other work during the day, such as a document assistant or rendering, if the scheduler or the team's calendar keeps the two apart.

Software and licences to check

On request the server arrives with the operating system, NVIDIA driver, CUDA and a container runtime, so a team can start PyTorch or Ultralytics containers on the first day. Ultralytics marks its configuration files with the AGPL-3.0 licence, and its licence page offers an Enterprise Licence for use without open-sourcing the whole project; which of the two fits a company's use of the code and the trained models is a question for its legal department. The COCO dataset is marked CC BY 4.0. Images from the company's own lines usually need no third-party licence, but may show people, which the data protection officer reviews.

Ordering a vision training server from us

You can buy this system from us: our team sizes the configuration to your workload and budget and replies within one business day. Sizing is free. The server is assembled and tested before delivery, and we deliver it anywhere in the EU on one contract and invoice, with manufacturer warranty on every component and the assembly covered by us; other destinations go through our logistics partner. Commissioning on site is available on request, and the MLOps platform on top is part of our Private AI/ML service. Quantities and lead time are confirmed in writing with the quote.

For an exact configuration we need the model families, image resolution, dataset size and growth, how many training jobs should run at once, how often models are retrained, and the power feed at the rack position. Describe these in the form below, and we reply with a configuration and a written quote.

FAQ