Qwen3.8-27B on NVIDIA RTX PRO 4500 Blackwell Server Edition: one card, pairs and users

Qwen3.8-27B on a 32 GB card

Qwen3.8-27B does not fit one NVIDIA RTX PRO 4500 Blackwell Server Edition in Qwen's FP8 version: its 28.7 GiB of weights exceed the 25.7 GiB our users-per-card rule leaves on a 32 GB card. NVIDIA's NVFP4 version of 20.4 GiB fits one card with room for a few conversations, and the FP8 version runs on a pair of cards with tensor parallelism, TP=2, for about 19 conversations of 32K tokens with an FP8 cache. For a department that runs mainly this model, one RTX PRO 6000 Server Edition holds more conversations than two pairs of the smaller card.

The RTX PRO 4500 Server Edition is the passive, single-slot data-centre card with 32 GB of GDDR7 at 800 GB/s and 165 W, described in our guide to the RTX PRO 4500 Blackwell Server Edition. Our pages on four cards in 2U and eight cards size Qwen3 models of the previous generation; this page answers the question for Qwen3.8-27B, the dense model with a vision encoder that Qwen published in August 2026.

Checkpoints of Qwen3.8-27B and their size

File lists of Qwen/Qwen3.8-27B-FP8 and nvidia/Qwen3.8-27B-NVFP4 on Hugging Face, read on 24 October 2026; BF16 size from our Qwen hardware guide. Fit by our users-per-card rule, 25.7 GiB per card. SE: Server Edition.

NVIDIA's card for the NVFP4 version names Blackwell as the supported microarchitecture and a GB300 as its test system. It quantises the MLP layers and the output layer to NVFP4 and the self-attention and linear-attention layers to FP8, and was made with Model Optimizer 0.48.0. Against BF16, NVIDIA reports 88.01 against 88.92 on GPQA Diamond, 74.86 against 75.14 on MMMU-Pro and 78.93 against 80.07 on IFBench. The page shows "18B params" in its metadata against 27B in the text. NVIDIA states that use is governed by Apache 2.0, the licence of the base model.

vLLM's Qwen3.8-27B recipe, updated on 15 October 2026, also lists third-party NVFP4 builds and states that the model "Fits one Blackwell GPU in every precision: NVFP4 in 24.6 GiB". A build of 24.6 GiB leaves about 1 GiB on a 4500 Server Edition by our rule, too little for users, while NVIDIA's leaves 5.3 GiB. On a 32 GB card the choice of NVFP4 build matters.

MIG does not help here. The card splits into two 1g.16gb halves, and a half that reports about 15.9 GiB holds 11.4 GiB of weights and cache by our rule, less than any of these checkpoints.

Cache per conversation, for text and documents

Qwen3.8-27B keeps a full KV cache in only 16 of its 64 layers. Its config.json gives those layers four KV heads of 256 values, 64 KiB per token in 16-bit, and the other 48 layers use Gated DeltaNet linear attention with a fixed state of 48 value heads of 128 × 128 values in float32, 144 MiB per conversation. Our Qwen hardware guide derives the figures.

Our arithmetic from config.json of Qwen/Qwen3.8-27B-FP8, read on 24 October 2026: 16 × 2 × 4 × 256 values per token plus 48 × 48 × 128 × 128 float32 values of linear-attention state per conversation.

The model reads images as well as text, which matters for document work. The vision encoder works on 16-pixel patches and merges 2 × 2 of them, so by our arithmetic each image token covers 32 × 32 pixels before any resizing by the image processor. An A4 page scanned at 150 dpi, 1,240 × 1,754 pixels, comes to about 2,100 tokens, and ten pages to about 21,000, beyond an 8K budget. Size document assistants at 32K. vLLM's recipe states that "Text serving is what this recipe covers and what has been verified", and NVIDIA's command for the NVFP4 version sets --mm-encoder-tp-mode data, which runs the vision encoder on every GPU of a split copy. Our guide to vision-language models for document AI covers OCR pipelines on smaller models.

Layouts and users on four and eight cards

Our estimates by the users-per-card rule, not measurements: concurrent conversations at full length, cache per conversation as in the table above. Usable memory 25.7 GiB per RTX PRO 4500 Server Edition, 51.4 GiB per pair, 102.8 GiB for four; 83.0 GiB per RTX PRO 6000 Server Edition. SE: Server Edition.

The pair is the arithmetic that decides. Two cards give 51.4 GiB; less 28.7 GiB of FP8 weights that leaves 22.7 GiB, and 22.7 ÷ 1.14 gives 19 conversations of 32K, or 10 with a 16-bit cache. One NVFP4 copy on one card leaves 5.3 GiB, which is 4 conversations of 32K. One RTX PRO 6000 Server Edition with the FP8 weights keeps 54.3 GiB for cache, 47 conversations of 32K, more than a four-card server with two pairs.

These are requests in flight at the same moment. With the example values of our guide to a private ChatGPT server by company size, 500 staff produce about 20 requests in flight at the peak, which one FP8 pair holds at 8K with room to spare and at 32K with one conversation short. vLLM's recipe sets an FP8 KV cache in most of its commands, as these figures assume.

We supply RTX PRO 4500 Server Edition cards and servers built around them; send us the model, context length and peak requests you plan, and our team replies with the layout and card count for Qwen3.8-27B.

FP8 blocks and FP4 on the Blackwell 4500

Qwen's FP8 version scales its weights in blocks of 128 × 128 with dynamic activation scales, as its config.json states, and keeps the vision encoder, the embeddings, the output layer and the linear-attention parameters unquantised. The RTX PRO 4500 Server Edition is a Blackwell card of the SM120 family, and for that family the engines document different paths:

• vLLM. The 27B recipe ran on two consumer Blackwell cards of the same chip family with TP=2 and states that "The block-scaled FP8 checkpoint needs no workaround." vLLM 0.30.0 of 22 September 2026 made native W4A4 NVFP4 the default over weight-only kernels on SM120 (pull request #55170).
• SGLang. Its quantisation documentation lists a CUTLASS kernel for blockwise FP8 on SM120 in the automatic choice, and FlashInfer CUTLASS for NVFP4 on SM120.
• TensorRT-LLM. Its quantisation matrix (1.4.0rc1, 19 October 2026) lists NVFP4 and per-tensor FP8 for SM120 but no FP8 block scaling, so Qwen's FP8 version is a vLLM or SGLang job on this card.

Measured throughput lags the hardware. StorageReview tested an NVFP4 build of Qwen3.8-27B on the card in a Dell PowerEdge R770 with NGC vLLM 26.08 (vLLM 0.27.1) and published the results on 14 October 2026. At 32 streams of 512 tokens in and out the card produced 320 output tokens per second, against 480 for an L40S, which has no FP4 Tensor Cores, and 616 for an RTX PRO 6000 Workstation Edition tested in a workstation; with 8,192 tokens in and 1,024 out it produced 153. A single stream ran at 17 to 22 tokens per second on every card. StorageReview measured 568.6 TFLOPS in NVFP4 against 280.1 in FP8 on the card's matrix units and concluded that "NVFP4 is a software story, on every card". Our RTX PRO 4500 benchmarks collect the other results. Test the vLLM version you will deploy before you count on FP4 throughput.

Tensor parallel TP=2: a pair over PCIe

The card has no NVLink, so a pair exchanges partial results over PCIe 5.0 at every layer. With exactly two GPUs, vLLM keeps its custom all-reduce if the peer-to-peer test at start-up passes; with four GPUs in one copy it falls back to NCCL by default, unless the opt-in FlashInfer PCIe IPC all-reduce that vLLM 0.30.0 added for systems without NVLink (release notes of 22 September 2026, #53576) is switched on. That makes two pairs on a four-card server the simpler layout and one TP=4 copy the layout with the most conversations. vLLM's recipe puts the FP8 weights at 14.28 GiB per GPU on two GPUs, in line with half of our 28.7 GiB. Our article on one model on several GPUs over PCIe or NVLink sets out the trade-off.

Bandwidth sets the ceiling for one conversation. A dense model reads its weights once per generated token; the FP8 checkpoint less its 2.5 GB embedding table (248,320 × 5,120 values in 16-bit) is about 28.3 GB. A pair at 2 × 800 GB/s therefore tops out near 56 tokens per second for one user before the all-reduce cost, about the same as one RTX PRO 6000 Server Edition at 1,597 GB/s, by our estimate. In a two-socket server, nvidia-smi topo -m shows which cards share a processor; a pair under one socket avoids the link between the sockets.

Desktops and a model in one server

Many buyers choose the 4500 Server Edition for virtual desktops as well as AI. NVIDIA's vPC sizing guide gives 16 desktops per card on 2 GB profiles, and the card needs vGPU 20 or later, on the R595 branch that NVIDIA supports until March 2027, as our guide to RTX PRO 4500 Server Edition vGPU profiles explains. Qwen3.8-27B in FP8 needs two whole cards, since NVIDIA's MIG guide states that NCCL is not supported with MIG. In a four-card server, one pair for the model leaves two cards for 32 desktops at that density. Whether desktops and the model run under one hypervisor host with your vGPU release is something we check with the configuration; for a fixed split, separate hosts are the cleaner design.

Power and rack position

At 165 W per card, four cards draw 660 W and, with two 280 W AMD EPYC 9355 and 15 per cent for the rest, the server about 1.4 kW at the wall by our estimate; eight cards about 2.2 kW. Both stay inside the 2.94 kW we plan per 230 V 16 A feed. One RTX PRO 6000 Server Edition draws up to 600 W on its own, about the same as four 4500 Server Edition cards, and a server with four of them needs about 3.4 kW. A measurement on the delivered system confirms these estimates. The 2U chassis that list four cards are on our page to buy a 4× RTX PRO 4500 Server Edition server.

When one RTX PRO 6000 Server Edition is the better buy

If Qwen3.8-27B is the main workload, one RTX PRO 6000 Server Edition holds the FP8 weights with 47 conversations of 32K by our rule, against 19 for a pair of 4500 Server Edition cards, without any traffic over PCIe and with the same single-stream ceiling. A 2U server with two of them, our entry configuration, holds about 94 as two copies. Our page on the two-card RTX PRO 6000 Server Edition server works it out.

The 4500 Server Edition is the better fit where a server carries desktops, MIG instances for small models and Qwen3.8-27B as one of several workloads, or where a 2U slot budget and a 16 A feed rule out the larger card. Our comparison of eight RTX PRO 4500 or four RTX PRO 6000 Server Edition weighs the two for whole servers, and the same model on 48 GB Ada cards is on our page about Qwen3.8-27B on the L40S. Full fine-tuning of the model belongs on H200 NVL, as our guide to full fine-tuning of Qwen3.8-27B shows.

What we supply

You can buy this system from us: our team sizes the configuration to your workload and budget and replies within one business day. We supply the RTX PRO 4500 Server Edition as a card and build AI servers to order with four or eight of them in chassis whose makers list them, assembled and tested before delivery, on one EU contract and invoice with manufacturer warranty on every component. vGPU and NVIDIA AI Enterprise licences come on the same invoice. Serving Qwen3.8-27B with retrieval on your documents is part of our Private AI/ML service; describe the documents and users you have in mind and we size the pairs.

FAQ