Full fine-tuning Qwen3.8-27B: four H200 NVL or eight RTX PRO 6000 Server Edition cards

Which server for which Qwen3.8-27B fine-tune

A full fine-tune of Qwen3.8-27B needs about 445 GB of GPU memory for model states alone, which four NVIDIA H200 NVL with a 4-way NVLink bridge or eight NVIDIA RTX PRO 6000 Blackwell Server Edition cards hold. Four H200 NVL keep all traffic on NVLink in one domain; eight Server Edition cards leave more memory per card but move the sharded weights over PCIe. LoRA and QLoRA on the same model need one card.

This page is about one model and the full fine-tune. Our fine-tuning server guide covers configurations for 8B to 70B dense models in general, and the best GPU for fine-tuning in 2026 compares the cards themselves.

Our arithmetic from the 55.6 GB of BF16 safetensors on the model's Hugging Face files page (read 24 October 2026): 16 bytes per parameter with 32-bit Adam and about 10 with 8-bit Adam (ZeRO paper's count), fully sharded with FSDP or ZeRO-3; offload keeps 14 bytes per parameter in host memory. Left is measured against the listed 141 GB and 96 GB per card. Activations come on top.

Qwen3.8-27B in numbers

Qwen's model card describes Qwen3.8-27B as a "Causal Language Model with Vision Encoder" with 27B parameters; Hugging Face's metadata lists 28B. The BF16 checkpoint on the files page consists of 18 safetensors shards that add up to 55.6 GB, which at 2 bytes per parameter gives about 27.8 billion parameters by our count. The card states a context of "262,144 natively and extensible up to 1,000,000 tokens" and the Apache 2.0 licence.

The language model has 64 layers with a hidden dimension of 5,120, arranged as "16 × (3 × (Gated DeltaNet → FFN) → 1 × (Gated Attention → FFN))". Three of every four layers use Gated DeltaNet, a linear attention, so only 16 layers keep a per-token KV cache; our Qwen hardware requirements page works out what that means for serving. For training, the layout matters in one practical way: the linear-attention layers need dedicated kernels to run fast. Unsloth's Qwen3.8 training page states that it "utilizes Flash Linear Attention kernels for efficient Qwen3.8 training" and asks for "the latest Transformers v5". Check that your framework lists the architecture, Qwen3_5ForConditionalGeneration in the model's config.json, before you plan a schedule.

The vision encoder in config.json has 27 blocks of width 1,152 with an MLP width of 4,304, which comes to about 0.4 billion parameters in attention and MLP weights by our arithmetic, under 2 per cent of the model.

Full fine-tuning: where 445 GB fit

By the ZeRO paper's accounting, mixed-precision training with Adam holds 2 bytes of weights, 2 bytes of gradients and 12 bytes of optimiser state per parameter. For 27.8 billion parameters that is 444.8 GB, or 500.4 GB by the 18-byte count in Hugging Face's documentation. No single card holds that, and two H200 NVL hold it neither with 32-bit nor with 8-bit Adam.

Four H200 NVL, 564 GB. Fully sharded, each card holds 111.2 GB of states and has about 30 GB for activations, the gathered layer and the CUDA context. That works with gradient checkpointing and moderate sequence lengths; long sequences or larger micro-batches push it to the edge. With 8-bit Adam the states drop to 69.5 GB per card and the room grows to about 72 GB.

Eight RTX PRO 6000 Server Edition, 768 GB. Each card holds 55.6 GB and has about 40 GB left, more headroom per card than four H200 NVL, at the price of traffic over PCIe, covered in the next section.

Four RTX PRO 6000 Server Edition, 384 GB. Too small with 32-bit Adam. With 8-bit Adam each card holds 69.5 GB and keeps about 27 GB. The other route is DeepSpeed's optimiser offload: the ZeRO-Offload paper keeps the 16-bit parameters on the GPU and 14 bytes per parameter in host memory, about 389 GB here, so each card holds 13.9 GB of weights and the 768 GB of host memory in our four-card reference configuration take the rest. The processors then perform the parameter update, which slows every step.

Eight H200 NVL, 1,128 GB. Holds the run at 55.6 GB per card with about 85 GB left, enough for long sequences, or two full fine-tunes of the 27B side by side, one per NVLink domain of four cards.

We build these servers with four or eight cards; send us the sequence length and the share of image data in your training set, and our team replies with the configuration and the memory left per card.

NVLink or PCIe: what sharding a 27B model moves

In a fully sharded run every card holds a slice of the weights and receives the rest twice per step, before the forward and again before the backward pass, and the gradients are reduce-scattered afterwards. The ZeRO paper puts this at 1.5 times the traffic of plain data-parallel training. By our arithmetic each card receives about 125 GB per forward and backward pass on four cards and about 146 GB on eight, in BF16.

Our arithmetic: 3 × 55.6 GB × (N − 1) / N received per card; link rates from NVIDIA (NVLink 900 GB/s per GPU, PCIe Gen5 128 GB/s, both totals for two directions). Compute: 6 × 27.8 billion × 8,192 tokens per card at NVIDIA's peak dense BF16 rate, 835.5 TFLOPS for the H200 NVL and 503.8 for the RTX PRO 6000 Workstation Edition; NVIDIA publishes no dense rate for the Server Edition, whose clocks are lower.

The comparison is a lower bound on both sides, since real runs reach neither the peak link rate nor the peak tensor rate. Two conclusions hold anyway. On four H200 NVL the transfer is a small fraction of the compute, so FSDP or ZeRO-3 can hide it behind the computation. On eight Server Edition cards the transfer is of the same order as the compute, and the step rate depends on how well the framework overlaps the two and on working peer-to-peer transfers between the cards. NVIDIA's NCCL documentation states that "CUDA and the NVIDIA driver stack do not support IOMMU-enabled PCIe peer-to-peer memory transfer", which our P2P guide for the RTX PRO 6000 turns into an acceptance check.

Raw compute favours the eight cards: eight times 503.8 TFLOPS at the Workstation Edition's rate is about 4,030 TFLOPS against about 3,342 for four H200 NVL, and the Server Edition runs somewhat below that rate. We found no published benchmark of a 27B full fine-tune on either configuration, so we give no measured ratio. More tokens per micro-batch help the PCIe build most, because the traffic per pass stays the same while the compute grows. A run on eight H200 NVL across both NVLink domains also sends part of its traffic over PCIe; our guide to the H200 NVL NVLink bridge shows how to see the domains with nvidia-smi topo -m.

LoRA and QLoRA on one card

Most teams that want a domain version of Qwen3.8-27B start with an adapter. A LoRA keeps the 55.6 GB base frozen in BF16 and trains rank-decomposition matrices. On all linear layers of the language model at rank 16 we count roughly 0.1 billion trainable weights from the layer sizes in config.json, about 1.9 GB with their gradients and Adam states at 16 bytes each. The run then holds about 57.5 GB, which leaves about 38 GB on an RTX PRO 6000 Server Edition and about 83 GB on an H200 NVL.

QLoRA stores the linear layers of the base in 4-bit NormalFloat at about 0.516 bytes per weight with double quantisation, and keeps the embeddings and the output layer in BF16. Qwen3.8-27B has a vocabulary of 248,320 tokens and an untied output layer, so these two alone are 2.54 billion parameters and 5.1 GB. With the vision tower in BF16 and the adapter, we arrive at about 21 GB. Unsloth's training page gives a matching figure: "QLoRA works with 24GB and LoRA needs >36GB". Its LoRA figure is below our arithmetic for a BF16 base, which already needs 55.6 GB for the weights; we plan LoRA on a 96 GB card.

On a server with four RTX PRO 6000 Server Edition cards, four LoRA runs train side by side, one per card, without any traffic between the cards. That is the cheapest way to test whether a domain adaptation works before committing to a full fine-tune. Our GPU memory guide for fine-tuning explains the per-parameter rules behind these figures.

The vision tower: freeze or train

Qwen3.8-27B is a vision-language model, and a fine-tune decides what happens to the image path. Unsloth's page states: "Set finetune_vision_layers=False when your dataset is text-only", and for image data it enables the vision layers with a conversational vision dataset. Training the 0.4 billion vision parameters adds about 6.6 GB of model states in a full fine-tune at 16 bytes per parameter, a small share of the 445 GB.

Images cost more as activations than as parameters. With a patch size of 16 pixels, a 1,024 × 1,024 image is split into 4,096 patches, which the encoder's merger combines two by two in each direction (spatial_merge_size 2 in config.json) into 1,024 image tokens, and these tokens lengthen every training sequence they appear in. For a dataset with many images per example, plan the sequence length and micro-batch with that in mind. A text-only fine-tune with the vision tower frozen keeps the image path of the original model.

Host memory, storage and power

NVIDIA's configuration guide for NVIDIA-Certified Systems asks for at least twice the GPU memory in host memory. Our reference builds give 1,152 GB for four H200 NVL and 1,536 GB for eight RTX PRO 6000 Server Edition cards, both well above the 389 GB that optimiser offload would need for this model.

Checkpoints set the storage. A resumable checkpoint of a full fine-tune holds the 32-bit master weights and Adam moments plus the 16-bit weights, 14 bytes per parameter by the ZeRO count, about 389 GB for Qwen3.8-27B by our estimate. Keeping the last three takes about 1.2 TB; the finished model is again a 55.6 GB BF16 checkpoint, which Qwen's FP8 version reduces to 30.9 GB for serving, as our Qwen page lists.

Our estimate: 600 W per card, two AMD EPYC 9355 (280 W) in the four-card server and two EPYC 9555 (360 W) in the eight-card servers, plus 15 per cent for memory, drives, fans and conversion losses; a measurement on the delivered system confirms the figure. We plan 80 per cent of a feed: 2.94 kW at 230 V 16 A, 5.89 kW at 32 A.

A full fine-tune keeps all cards near full power for the length of the run, so plan the feed for the full load. At a 450 W cap the four-card H200 NVL server comes down to about 2.7 kW by the same estimate, at a lower step rate.

Recommendation

Domain adaptation, first project. LoRA or QLoRA on one card; a server with two or four RTX PRO 6000 Server Edition cards runs several experiments at once and serves the result. See our page to buy a 4× RTX PRO 6000 Server Edition server.

Full fine-tuning of the 27B as the plan. Four H200 NVL with a 4-way bridge and 1,152 GB of host memory: the whole run stays inside one NVLink domain. Our page to buy a 4× H200 NVL server lists the build.

Full fine-tuning with long sequences, or the same server for serving many users later. Eight RTX PRO 6000 Server Edition cards give 40 GB per card of headroom and more aggregate compute over PCIe; eight H200 NVL give the most headroom on NVLink. Our pages to buy an 8× RTX PRO 6000 Server Edition server and to buy an 8× H200 NVL server cover both.

You can buy this system from us: our team sizes the configuration to your workload and budget and replies within one business day. The server is assembled and tested before delivery and delivered in the EU on one contract and invoice, with manufacturer warranty on every component; quantities and lead time are confirmed in writing with the quote. The training pipeline and serving the tuned model are part of our Private AI/ML service with our engineering partner Vixen.UNO. Write to us with your dataset and the method you plan, and we compare both builds for your run.

FAQ