Qwen3.8-27B on NVIDIA L40S and RTX 6000 Ada: FP8 on Ada and users per card

Qwen3.8-27B on a 48 GB Ada card

Qwen3.8-27B runs on one NVIDIA L40S in Qwen's FP8 version: the 28.7 GiB of weights fit the card's 48 GB and leave 8.6 GiB by our users-per-card rule with ECC on. That is room for 7 concurrent conversations of 32,768 tokens with an FP8 cache, or 21 of 8,192 tokens, so one card serves a team and a fleet of cards serves a department. The checkpoint's block-scaled FP8 runs on Ada through Triton kernels in vLLM and SGLang; we found no published throughput figure for that path on this model.

This page is for companies with L40S servers or RTX 6000 Ada workstations that want to run the current Qwen dense model on them. The same model on the 32 GB Blackwell card, where it needs two cards in FP8, is on our page about Qwen3.8-27B on the RTX PRO 4500 Server Edition, which also derives the cache per conversation. What differs on Ada is the memory with ECC, the FP8 kernels and the absence of FP4.

Memory on the L40S with ECC on and off

The L40S carries 48 GB of GDDR6 with ECC at 864 GB/s, 733 TFLOPS of dense FP8 and 350 W, on PCIe 4.0 x16, without NVLink or MIG. NVIDIA's CUDA Best Practices Guide states that ECC on GDDR memory takes 6.25 per cent of capacity. Scaled from the 97,887 MiB a 96 GB card reports, a 48 GB card shows about 47.8 GiB, or 44.8 GiB with ECC on. Our rule takes 90 per cent and subtracts 3 GiB, which leaves 37.3 GiB with ECC on and 40.0 GiB with ECC off, as our page on the eight-card L40S server works out. Check the figure with nvidia-smi -q in the ECC mode you run.

Our estimates by the users-per-card rule, not measurements, for one L40S: weights 28.7 GiB (30.9 GB) in FP8, 20.4 GiB (21.9 GB) in NVIDIA's NVFP4 build and 51.8 GiB (55.6 GB) in BF16, from Hugging Face file lists read on 24 October 2026; cache 0.39 GiB per 8K and 1.14 GiB per 32K conversation in FP8, 2.14 GiB per 32K in 16-bit, including the fixed linear-attention state.

The FP8 cache matters more on a 48 GB card than on a 96 GB one. With a 16-bit cache, one card holds only four conversations of 32K, which suits a pilot, not a service. vLLM sets the FP8 cache with --kv-cache-dtype fp8, and its Qwen3.8-27B recipe uses it in most commands. Switching ECC off adds 2.7 GiB of cache per card at the cost of error correction on the memory; for a production service we keep ECC on and size by the first row.

How Qwen's block-scaled FP8 runs on Ada

The L40S, the L4 and the RTX Ada cards are compute capability 8.9 and have FP8 Tensor Cores, so FP8 is the native precision of this generation. Qwen's FP8 version is a particular kind of FP8. Its config.json sets e4m3 values, dynamic activation scales and a weight_block_size of 128 × 128, and keeps the vision encoder, the embeddings, the output layer and the linear-attention parameters unquantised. Each block of the weight matrices carries its own scale, which needs kernels that apply those scales inside the matrix multiplication.

What the engines document for that format on Ada, as of October 2026:

• SGLang. Its quantisation page lists the blockwise FP8 kernels by chip: DeepGEMM for SM90 and SM100, FlashInfer TensorRT-LLM for SM100, CUTLASS for SM120, and Triton as the "Fallback; widely compatible" for all. Its automatic choice runs in that order, so on SM89 it lands on Triton.
• vLLM. Its FP8 utilities contain a Triton kernel for block-scaled FP8 matrix multiplication, w8a8_triton_block_scaled_mm. vLLM's Qwen3.8-27B recipe, updated on 15 October 2026, lists test systems with consumer Blackwell, Intel and Ascend hardware and none with Ada.
• TensorRT-LLM. Its quantisation matrix lists only per-tensor FP8 for Ada, so Qwen's block-scaled version is not on its supported list for this generation.

The model's architecture runs on Ada in vLLM: StorageReview served an NVFP4 build of Qwen3.8-27B on the L40S with vLLM 0.27.1, as the next section shows. We found no published measurement of Qwen's FP8 version on an Ada card, so the speed of the Triton path for this model is a figure for a test run, not for a datasheet.

Community FP8 builds with per-channel scales exist on Hugging Face for exactly this reason. One such card, which describes itself as "a community quantization, not an official Qwen release", states that per-channel FP8 uses CUTLASS on SM 8.9 while block-128 "falls back to a Triton kernel". It keeps the linear-attention layers in BF16 and comes to about 37 GB, which on an L40S with ECC on leaves about 2.9 GiB, two conversations of 32K by our rule. Qwen's own version leaves three times as much cache, which for most fleets outweighs a faster kernel.

NVFP4 on an Ada card: weight-only

Ada has no FP4 Tensor Cores. vLLM loads NVFP4 checkpoints on these cards as weight-only W4A16 through its Marlin kernels and logs a warning that this may reduce throughput for compute-heavy work, as our article on FP4 models on H200 NVL and Ada GPUs sets out. The weights stay in 4-bit in memory and are converted to 16-bit inside each multiplication.

NVIDIA's NVFP4 build of Qwen3.8-27B, 21.9 GB, names Blackwell as its supported microarchitecture. On an L40S it would leave 16.9 GiB for cache with ECC on, twice the room of the FP8 version, 14 conversations of 32K by our rule. That makes it worth a test on Ada fleets that are short of memory, with the caveat that NVIDIA does not list the generation for it.

StorageReview's measurement points the same way. In its roundup of 14 October 2026, with NGC vLLM 26.08 (vLLM 0.27.1) in a Dell PowerEdge R770, the L40S, which has no FP4 Tensor Cores, reached 480 output tokens per second on an NVFP4 build of Qwen3.8-27B at 32 streams of 512 tokens in and out "by dequantizing", against 320 for the RTX PRO 4500 Server Edition and 616 for an RTX PRO 6000 Workstation Edition, which StorageReview tested in a workstation. With 8,192 tokens in and 1,024 out the L40S reached 190. A single stream ran at 17 to 22 tokens per second on every card. StorageReview did not name the NVFP4 build it used. Our L40S benchmark review collects the other published figures for the card.

vLLM 0.32.0 of 18 October 2026 added an NVFP4 KV cache on SM8x. For a hybrid model with linear-attention state we have not sized it, and it belongs in the same test run.

We supply L40S cards and servers and add cards to existing fleets; send us your current servers, the context length and the peak requests you plan, and our team replies with a layout for Qwen3.8-27B.

Eight copies or four pairs on eight L40S

A server with eight L40S can run eight independent copies, one per card, or four copies split over two cards each with tensor parallelism, TP=2. A pair holds 74.6 GiB with ECC on; less 28.7 GiB of weights that leaves 45.9 GiB, and 45.9 ÷ 1.14 gives 40 conversations of 32K per pair.

Our estimates by the users-per-card rule, not measurements: concurrent conversations at full length with an FP8 cache, ECC on unless stated; usable memory 37.3 GiB per card with ECC on, 40.0 GiB with ECC off. The NVFP4 row assumes the weight-only path described above.

Pairs hold almost three times as many conversations, because the weights are stored four times instead of eight. They pay with traffic over PCIe 4.0 at every layer. With exactly two GPUs per copy, vLLM keeps its custom all-reduce if the peer-to-peer test passes, and vLLM's parallelism guide names the L40S among GPUs without NVLink, for which it advises pipeline parallelism over tensor parallelism. In a two-socket server, nvidia-smi topo -m shows which cards share a processor; keep each pair under one socket. Our article on one model on several GPUs over PCIe or NVLink covers the choice.

Eight copies have their own advantage: a failed card takes one eighth of the capacity out of service, and every copy restarts on its own. A mixed layout, for example two pairs for Qwen3.8-27B and four cards for an embedding model, a reranker and smaller models, often fits a department better than one layout for all eight cards, as our guide to serving several models on one GPU server explains.

Speed per user on the L40S

A dense model reads all its weights for every generated token. Qwen's FP8 checkpoint less its 2.5 GB embedding table is about 28.3 GB, so by our estimate one L40S at 864 GB/s tops out near 30 tokens per second for a single user, an RTX 6000 Ada at 960 GB/s near 34 and an ideal TP=2 pair of L40S near 61 before the cost of the all-reduce. Real figures land below these ceilings, and with many users the batch shares each read of the weights.

Qwen3.8-27B thinks before it answers by default, as its model card states, so answers carry more tokens than a plain chat model's. At 30 tokens per second a reasoning block of 1,000 tokens takes more than half a minute before the answer starts. For interactive use, set the thinking mode per request, or choose pairs, which raise the speed per user as well as the number of conversations.

RTX 6000 Ada workstations

The RTX 6000 Ada has the same Ada core configuration and the same 48 GB of GDDR6 with ECC as the L40S, with an active cooler, 300 W and 960 GB/s. The memory figures and layouts above therefore apply unchanged, and the single-stream ceiling is about 11 per cent higher. A workstation with two cards holds one TP=2 pair with 40 conversations of 32K by our rule, or two copies with seven each, for a team that develops on the model. Our comparison of the RTX 6000 Ada and the L40S explains where each card belongs, and our page to buy an RTX 6000 Ada workstation lists the configurations.

Power for four and eight L40S

At 350 W per card, four L40S draw 1,400 W and, with two 280 W AMD EPYC 9355 and 15 per cent for the rest of the server, about 2.3 kW at the wall by our estimate, which one 230 V 16 A feed per side carries. Eight cards draw 2,800 W and the server about 3.9 kW, which needs a 32 A single-phase or a three-phase feed per side. A measurement on the delivered system confirms both figures. The configurations and chassis are on our page to buy an L40S server.

Adding Blackwell cards to an Ada fleet

Where Qwen3.8-27B has to serve many users at long context, a 96 GB card changes the arithmetic. One RTX PRO 6000 Server Edition holds the FP8 weights with 47 conversations of 32K by our rule, more than a pair of L40S, computes NVFP4 natively and has MIG. Our comparison of the L40S and the RTX PRO 6000 Server Edition weighs the two, and our guide to mixing Ada and Blackwell GPUs covers drivers, CUDA versions and scheduling when both generations run in one estate. For Ada fleets the practical path is to keep Qwen's FP8 version on the L40S servers, test the NVFP4 build for memory, and place new long-context work on Blackwell.

What we supply

You can buy this system from us: our team sizes the configuration to your workload and budget and replies within one business day. We supply the L40S and the RTX 6000 Ada as cards, build AI servers to order with four or eight L40S in chassis whose makers list them, assembled and tested before delivery, on one EU contract and invoice with manufacturer warranty on every component. We also supply the RTX PRO 6000 Server Edition and RTX PRO 4500 Server Edition for fleets that move to Blackwell.

FAQ