L4 vs L40S: which inference card for the servers you already have

Two Ada cards for two different jobs

The L4 is a 72 W low-profile card with 24 GB for 7B to 14B models, video streams and light virtual desktops; the L40S is a 350 W full-size card with 48 GB for models up to about 32B in FP8, graphics and more throughput per card. NVIDIA launched the L4 in March 2023 as a low-power inference and video card, comparing it with the T4 as the previous GPU generation, and the L40S later that year as its “universal” data-centre GPU for inference, training, graphics and video. Both use the Ada Lovelace architecture with FP8 Tensor Cores, both are passively cooled and depend on the server’s fans, both support secure boot with a hardware root of trust, and both are on NVIDIA’s vGPU list. Neither supports MIG or NVLink. They differ in size and power: the L4 draws 72 W from the slot in a low-profile single-slot format, the L40S up to 350 W, with a 16-pin auxiliary connector, in a full-size dual-slot card.

L4 vs L40S specifications

NVIDIA L4 and L40S product pages, datasheets and product briefs, NVIDIA’s Ada architecture whitepaper and Lenovo’s L4 product guide; NVIDIA vGPU supported-GPU list. NVIDIA’s sparse figures assume structured sparsity; its dense figures are about half.

MLPerf® results, divided per card

MLPerf® Inference v5.1, published in September 2025, includes a benchmark in which both cards were tested in the same round and division: RetinaNet object detection in the datacenter category of the closed division, offline scenario, with every system running TensorRT.

MLPerf® Inference v5.1, datacenter category, closed division, RetinaNet, offline scenario; entries as listed, retrieved from mlcommons.org on 24 September 2026, each result verified by MLCommons Association. Per card is our division of the system result by its number of GPUs, not an MLPerf® metric. The MLPerf name and logo are registered and unregistered trademarks of MLCommons Association in the United States and other countries. All rights reserved. Unauthorized use strictly prohibited. See www.mlcommons.org for more information.

Per card, by our division, the L40S in the eight-card system does about 3.6 times the work of an L4 in the two-card system, in the same round, division and scenario. These runs were not in the MLPerf® power category and measured no power, so they do not show which card does more work per watt.

For language models the L40S has the published results. In MLPerf® Inference v5.0, published in April 2025, a Dell PowerEdge XE7745 with eight L40S (entry 5.0-0018) processed 3,481.5 tokens per second on Llama 2 70B (99 per cent accuracy variant) in the offline scenario, about 435 per card. The same server model with eight H200 NVL (entry 5.0-0017) reached 31,149.9 in the same round, division and scenario, about 3,890 per card. Both are datacenter results in the closed division, retrieved from mlcommons.org on 24 September 2026, each result verified by MLCommons Association; per card is our division of the system result by its number of GPUs, not an MLPerf® metric. We found no comparable published language-model result for the L4 in the datacenter results of recent rounds.

Which models fit in 24 GB and 48 GB

A 24 GB L4 takes an 8B model in FP8, gpt-oss-20b at 13.8 GB, or a 14B model in FP8 with a modest cache. A 48 GB L40S takes a 32B model in FP8, but by our users-per-card rule (0.9 × the memory the driver reports, less 3 GiB of overhead, less the weights) that leaves cache for about two concurrent 8K conversations with a 16-bit cache and five with an FP8 cache; a 70B model in 4-bit, about 40 GB of weights, leaves almost nothing for the cache. Neither card holds gpt-oss-120b on its own, and neither has FP4 arithmetic: Ada runs 4-bit weights as INT4 through AWQ or GPTQ, which TensorRT-LLM supports on both cards; vLLM also loads NVFP4 checkpoints on Ada, but only as weight-only compression with 16-bit arithmetic, as our FP4 article explains. Bandwidth then sets the speed: for the same model, token generation on an L40S can run up to about 2.9 times faster than on an L4 for a single user, the ratio of their bandwidths; that is a ceiling from the specifications, not a measurement.

If the model needs more than 48 GB, neither card is the answer. The RTX PRO 6000 Server Edition with 96 GB and the H200 NVL with 141 GB are the next steps, and our VRAM guide has the arithmetic.

We supply all four cards. Tell us the model and the number of concurrent users, and we work out which one your servers need and how many.

Video, graphics and virtual desktops

NVIDIA built the L4 for video as much as for inference. It has two encoders, four decoders and four JPEG decoders against three and three on the L40S, and both encode AV1. NVIDIA quotes up to 1,040 concurrent AV1 streams at 720p30 for a server with eight L4 cards, about 130 per card, and 120 times the AI video performance of a dual-socket CPU server in an end-to-end pipeline; both figures come from eight-card servers, and only the second is a comparison with a CPU-only server. For video analytics, transcoding and camera streams, the L4 has far more video engines per watt than the L40S.

For graphics, the advantage is with the L40S, which has 142 RT cores and display outputs that can be switched on; NVIDIA positions it for graphics and video as well as AI. For virtual desktops both work: the L40S supports up to 32 users per card with 1 GB profiles, the L4 up to 24 with 1 GB profiles, and both need NVIDIA vGPU licences. Every profile with its limits is listed in our guides to L40S vGPU profiles and L4 vGPU profiles.

We supply the vGPU licences together with the cards, in the same quote. Write to us with the number of concurrent desktop users and the applications they run, and we size the cards and recommend the licence edition and term.

Density and power per server

HPE, Dell and Lenovo product pages and product guides; maximum counts as the makers state them, September 2026.

The rack arithmetic follows. Eight L4 draw 576 W and sixteen draw 1,152 W; four L40S draw 1,400 W and eight draw 2,800 W, all before processors and fans. In the same Dell PowerEdge XE7745, sixteen L4 and eight L40S both add up to 384 GB of GPU memory; the L40S set has 6,912 GB/s of aggregate bandwidth against 4,800 GB/s and draws about 2.4 times the power. Sixteen small cards suit many small models or many streams; eight large ones suit fewer, larger models. Our article on rack power and cooling covers the room side.

When to choose the L4 or the L40S

The L4 fits when the work is many small models or many streams: video analytics, transcoding, speech, embeddings, 7B to 14B models, and virtual desktops for light users. It is also the card for servers without GPU power cables, provided the server maker lists the L4 for that chassis and slot, since the card is passive and relies on the server’s airflow, and for racks where every watt is counted.

The L40S fits when a model needs 24 to 48 GB, when throughput per card matters more than per watt, and when the same servers also run graphics or rendering.

Neither fits a 70B model serving a team, or anything larger; that needs an RTX PRO 6000 or an H200 NVL. One RTX PRO 6000 holds Llama 3.3 70B in FP8 with cache for six concurrent 8K sessions, twelve with an FP8 cache, by the same rule.

Lifecycle and what comes next

We found no end-of-life notice from NVIDIA for either card, and NVIDIA’s vGPU lifecycle page, last updated on 29 October 2026, lists the L4 and the L40S among the GPUs that “remain fully supported”. That page covers vGPU software support, not how long the cards remain orderable. NVIDIA compares two Blackwell cards with them. For the RTX PRO 6000 Blackwell Server Edition, which it sets against the L40S, its datasheet says the Tensor Cores “deliver up to 5x the performance of the previous generation for LLM inference with support for FP4 precision”; the L40S has no FP4. Against the L4 it sets the RTX PRO 4500 Blackwell Server Edition, a 165 W single-slot, full-height card with 32 GB and 800 GB/s that NVIDIA says delivers “over 5x the performance of the previous-generation L4 GPU”. We found no published test conditions for either claim, and the RTX PRO 4500 is not a low-profile card, so it does not fit where an L4 does. Our comparison of the L4 and the RTX PRO 4500 Server Edition covers that choice per server.

What we supply

For inference, video and virtual desktops, we supply the NVIDIA L4 and the L40S, as cards for servers you already run or in AI servers built to order, on one EU contract and invoice with manufacturer warranty. When a model outgrows 48 GB, we supply the RTX PRO 6000 Server Edition or the H200 NVL, as cards or in servers built to order, among them the inference node with four RTX PRO 6000 Server Edition cards (4 × 96 GB) in our AI server configurations. Before you order, we tell you which card fits your chassis and slot and how many of them the rack can power.

FAQ