L4 vs L40S: which inference card for the servers you already have

Two Ada cards for two different jobs

NVIDIA launched the L4 in March 2023 as a low-power inference and video card, the successor to the T4, and the L40S later that year as its “universal” data-centre GPU for inference, training, graphics and video. Both use the Ada Lovelace architecture with FP8 Tensor Cores, both are passively cooled and depend on the server’s fans, both support secure boot with a hardware root of trust, and both are on NVIDIA’s vGPU list. Neither supports MIG or NVLink. What separates them is scale: one draws 72 W from the slot in a low-profile single-slot format, the other up to 350 W, with a 16-pin auxiliary connector, in a full-size dual-slot card.

The specification table

NVIDIA L4 and L40S product pages, datasheets and product briefs, NVIDIA’s Ada architecture whitepaper and Lenovo’s L4 product guide; NVIDIA vGPU supported-GPU list. NVIDIA’s sparse figures assume structured sparsity; its dense figures are about half.

MLPerf® results, divided per card

MLPerf Inference v5.1, published in September 2025, includes a benchmark in which both cards and the H200 NVL were tested in the same round and division: RetinaNet object detection in the datacenter category of the closed division, offline scenario, with every system running TensorRT.

MLPerf Inference v5.1, datacenter category, closed division, RetinaNet, offline scenario; entries as listed, retrieved from mlcommons.org on 24 September 2026, each result verified by MLCommons Association. Per card is our division of the system result by its number of GPUs, not an MLPerf metric. The MLPerf name and logo are registered and unregistered trademarks of MLCommons Association in the United States and other countries. All rights reserved. Unauthorized use strictly prohibited. See www.mlcommons.org for more information.

Per card, by our division, the L40S in the eight-card system does about 3.6 times the work of an L4 in the two-card system, in the same round, division and scenario. These runs were not in the MLPerf power category and measured no power, so they do not show which card does more work per watt.

For language models the L40S has the published results. In MLPerf Inference v5.0, published in April 2025, a Dell PowerEdge XE7745 with eight L40S (entry 5.0-0018) processed 3,481.5 tokens per second on Llama 2 70B (99 per cent accuracy variant) in the offline scenario, about 435 per card; the same server model with eight H200 NVL (entry 5.0-0017) reached 31,149.9 in the same round, division and scenario, about 3,890 per card. Both are datacenter results in the closed division, retrieved from mlcommons.org on 24 September 2026, each result verified by MLCommons Association; per card is our division of the system result by its number of GPUs, not an MLPerf metric. We found no comparable published language-model result for the L4 in the datacenter results of recent rounds.

Memory decides the model

A 24 GB L4 takes an 8B model in FP8, gpt-oss-20b at 13.8 GB, or a 14B model in FP8 with a modest cache. A 48 GB L40S takes a 32B model in FP8 with room for users, or a 70B model in 4-bit with very little. Neither card holds gpt-oss-120b on its own, and neither has FP4 arithmetic: Ada runs 4-bit weights as INT4 through AWQ or GPTQ, which TensorRT-LLM supports on both cards, but not NVFP4. Bandwidth then sets the speed: for the same model, token generation on an L40S can run up to about 2.9 times faster than on an L4 for a single user, the ratio of their bandwidths; that is a ceiling from the specifications, not a measurement.

If the model needs more than 48 GB, neither card is the answer. The RTX PRO 6000 Server Edition with 96 GB and the H200 NVL with 141 GB are the next steps, and our VRAM guide has the arithmetic.

Video, graphics and virtual desktops

Video is where the L4 was designed to lead. It has two encoders, four decoders and four JPEG decoders against three and three on the L40S, and both encode AV1. NVIDIA quotes up to 1,040 concurrent AV1 streams at 720p30 for a server with eight L4 cards, about 130 per card, and 120 times the AI video performance of a dual-socket CPU server in an end-to-end pipeline; both figures come from eight-card servers, and only the second is a comparison with a CPU-only server. For video analytics, transcoding and camera streams, the L4’s engine count per watt is hard to beat.

Graphics goes the other way. The L40S has 142 RT cores and display outputs that can be switched on, and NVIDIA positions it for graphics and video as well as AI. For virtual desktops both work: the L40S supports up to 32 users per card with 1 GB profiles, the L4 up to 24 with 1 GB profiles, and both need NVIDIA vGPU licences.

Density and power per server

HPE, Dell and Lenovo product pages and product guides; maximum counts as the makers state them, September 2026.

The rack arithmetic follows. Eight L4 draw 576 W and sixteen draw 1,152 W; four L40S draw 1,400 W and eight draw 2,800 W, all before processors and fans. In the same Dell PowerEdge XE7745, sixteen L4 and eight L40S both add up to 384 GB of GPU memory; the L40S set has 6,912 GB/s of aggregate bandwidth against 4,800 GB/s and draws about 2.4 times the power. Sixteen small cards suit many small models or many streams; eight large ones suit fewer, larger models. Our article on rack power and cooling covers the room side.

When to order which

The L4 fits when the work is many small models or many streams: video analytics, transcoding, speech, embeddings, 7B to 14B models, and virtual desktops for light users. It is also the card for servers without GPU power cables, provided the server maker lists the L4 for that chassis and slot, since the card is passive and relies on the server’s airflow, and for racks where every watt is counted.

The L40S fits when a model needs 24 to 48 GB, when throughput per card matters more than per watt, and when the same servers also run graphics or rendering.

Neither fits a 70B model for a team, or anything larger: that is RTX PRO 6000 or H200 NVL territory.

Lifecycle and what comes next

We found no end-of-life notice from NVIDIA for either card, and NVIDIA’s vGPU lifecycle page of 3 August 2026 lists the L4 and the L40S among the GPUs that “remain fully supported”. That page covers vGPU software support, not how long the cards remain orderable. NVIDIA positions two Blackwell cards as their successors. Against the L40S it sets the RTX PRO 6000 Blackwell Server Edition, with “up to 5x higher large language model (LLM) inference throughput”. Against the L4 it sets the RTX PRO 4500 Blackwell Server Edition, a 165 W single-slot, full-height card with 32 GB and 800 GB/s that NVIDIA says delivers “over 5x the performance of the previous-generation L4 GPU”. Neither claim comes with published test conditions, and the RTX PRO 4500 is not a low-profile card, so it does not fit where an L4 does.

What we supply

Eurokommerz supplies the NVIDIA L4 and the L40S EU-wide with manufacturer warranty, as cards or in servers built to order, and the RTX PRO 6000 Server Edition when a model outgrows them. Send us the workload and the servers you run, and we will tell you which card fits and how many of them the rack can power.

FAQ