Securing a GPU server: the BMC, firmware, NVIDIA drivers, the Container Toolkit, inference APIs, model files and shared GPUs

The attack surface, layer by layer

A GPU server adds layers to the usual checklist, and two of them sit below the operating system: CISA describes baseboard management controllers as components that “operate separately from the operating system (OS) and firmware to allow for remote management and control, even when the system is shut down”. Versions and dates are as of October 2026.

Our summary of the sections below; sources: CISA, AMI, NVIDIA, vLLM, Ollama, PyTorch and Hugging Face documentation and CVE records, read in September and October 2026.

BMC and firmware

In a June 2025 technical blog, NVIDIA warns that compromised BMCs “become a platform for persistence across every system they control”. Its first recommendations are “Place BMC interfaces on isolated management networks and never expose them to the Internet” and “Work with your vendors to ensure BMC firmware is updated and CVEs are tracked”. It also advises treating BMC events as part of logging and detection.

Isolation does not replace patching. CVE-2024-54085, in AMI’s MegaRAC SPx BMC firmware, lets an attacker, in the record’s words, “bypass authentication remotely through the Redfish Host Interface”. AMI scores it 10.0 under CVSS 4.0, and its advisory of 11 March 2025 names fixed SPx versions; CISA added it to its Known Exploited Vulnerabilities Catalog on 25 June 2025, “based on evidence of active exploitation”. Ask your server maker which BMC release contains the fix. IPMI brings an older risk: the IPMI 2.0 specification itself lets a remote attacker obtain a password hash from the BMC for offline guessing (CVE-2013-4786). Where Redfish, which runs over HTTPS, covers your needs, we would switch IPMI over LAN off; every BMC needs a long, unique password.

The same AMI advisory also lists CVE-2024-54084, a BIOS flaw in its Aptio V firmware. AMI’s advisories of 8 September and 13 October 2026 list further Aptio V flaws, among them CVE-2026-33197, scored 8.7 by AMI, so ask the server maker for BIOS updates as well as the BMC release. GPUs carry firmware too: in NVIDIA’s H200 NVL product brief, the card’s CEC device “authenticates the contents of the GPU firmware ROM before permitting the GPU to boot from its ROM”, with rollback protection and key revocation, and the L40S product page and RTX PRO 6000 Server Edition datasheet list secure boot. Record each card’s VBIOS Version, which nvidia-smi -q prints, and take GPU firmware only from the server maker or NVIDIA.

NVIDIA driver security fixes per branch

NVIDIA publishes its security bulletins in its product-security repository on GitHub, as Markdown, machine-readable CSAF and CVE records: the repository states that, starting on 1 October 2026, NVIDIA PSIRT “will only publish security bulletins on GitHub in Markdown, CSAF, and CVE formats”. Subscribe to the repository for notifications of new or revised bulletins, and to the email updates offered on NVIDIA’s Product Security page.

NVIDIA’s lifecycle table gives a production branch “Quarterly (or as-needed) bug and security releases for 1 year”, and a long term support branch the same for 3 years. CVE-2026-47489, from NVIDIA’s security bulletin of 30 September 2026, is an example. This flaw in the kernel mode layer of the Linux display driver, scored 7.8 by NVIDIA, is fixed in 595.91.07 on R595 and 580.178.04 on R580, and on the New Feature Branch R615 in 615.71.09; the record lists no driver for R535, a branch whose end of life NVIDIA’s table gives as June 2026. The newer releases of 19 to 22 October 2026, 615.71.10, 580.196.07 and 595.112.07, name no CVE in their release notes. A server takes the security release of its own supported branch; our driver guide shows how to pin one. The record also lists vGPU guest drivers and the vGPU Manager up to vGPU 20.1 and 19.5, so the vGPU Manager on the host and the driver in each guest both need the fixed release, vGPU 20.2 or 19.6. The attack vector is local, and by our reading that can include code in a container given a GPU, because GPU containers use the host’s driver. We build AI servers to order and install the operating system, drivers, CUDA and a container runtime on request; write to us with the driver branch you want to pin and our team will help.

The Container Toolkit and the GPU Operator

The NVIDIA Container Toolkit runs on the host and sets up each GPU container as it starts; on Kubernetes, the GPU Operator deploys it by default. Five flaws, two scored 9.0 by NVIDIA, show why its version matters.

CVE records published by NVIDIA (cve.org and NVD), NVIDIA’s GitHub advisory for CVE-2024-0132, and the release notes of Container Toolkit 1.20.1 and GPU Operator 26.7.2, read on 23 October 2026; the Toolkit notes for 1.19 and 1.20 name no CVE. 1.17.8 is the first Toolkit release outside the records’ affected range, and GPU Operator 25.3.1 ships it; the Operator’s notes first name CVE-2025-23266 and CVE-2025-23267 in 25.3.2. CVE-2026-24260 was published on 1 July 2026; Toolkit 1.19.1 and GPU Operator 26.3.2, which ships it, are the first releases outside its affected range, and neither release note names the CVE.

CDI (the Container Device Interface) protected against CVE-2024-0132 and CVE-2025-23359 but not against the hook flaws, and three of the five descriptions name a crafted image as the means of attack. Pull images only from a registry you control, pin them by digest, and limit who may start GPU containers; on Kubernetes, that is anyone allowed to create pods on GPU nodes. The device plugin’s README, which assumes the NVIDIA runtime as the node’s default, warns that a pod requesting no GPU gets “all the GPUs on the machine”, so check that default as well. Since v25.10.0 the Operator injects GPUs through CDI by default and no longer makes the nvidia runtime class the default handler, and its 26.7 releases include Toolkit 1.20. An upgrade through OLM (Operator Lifecycle Manager) on OpenShift leaves cdi.enabled as it was, and NVIDIA’s release notes say that GPU management containers taking GPUs through NVIDIA_VISIBLE_DEVICES instead of the device plugin must set runtimeClassName: nvidia; allow that class only for trusted pods.

Inference APIs and notebooks

vLLM listens on all interfaces unless --host is set, and its --api-key protects only paths under /v1, /v2, /inference and /cohere. Its security page, read on 23 October 2026, lists the routes the key does not cover: inference routes such as /invocations, /pooling, /classify, /score and /rerank, and control routes such as /pause, /resume and /update_weights, which “can alter model behavior”. The page says not to rely on the key alone. The documentation’s answer is a reverse proxy that allowlists “only the endpoints you want to expose to end users”, and a firewall that admits only the API port; the ports for torch.distributed and KV cache transfer belong to trusted hosts, and traffic between nodes is “insecure by default”. Never set VLLM_SERVER_DEV_MODE=1 in production: it adds /collective_rpc, which the documentation calls “extremely dangerous”.

Ollama “binds 127.0.0.1 port 11434 by default”, so only local processes reach it, and OLLAMA_HOST changes that. Its official container image, however, sets OLLAMA_HOST=0.0.0.0:11434, and by default Docker publishes a port given as -p 11434:11434, as in Ollama’s Docker instructions, on all host addresses. Its local API “does not require authentication”, so such a container answers anyone who can reach the port. The same applies to a server opened with that setting, as one of NVIDIA’s DGX Spark playbooks does for remote access. Under CVE-2026-103663, published on 8 October 2026, an unauthenticated request to /api/pull could write a file outside the model store; where the Ollama service can write to /usr/lib/ollama, the file runs as root after a restart, the case the CVE record rates critical (CVSS 9.4), otherwise 6.9; Ollama 0.35.0 fixes it, so run 0.35.0 or later. Jupyter Server turns token authentication on by default, and its documentation explains why: “access to the Jupyter Server means access to running arbitrary code”. Our engine comparison covers the listening defaults of three more engines.

Whichever engine or notebook server you run, bind it to localhost or an internal address and put a TLS gateway in front that gives each team or application its own key, allowlists routes, limits rates and logs who asked what. Prompts and answers can contain personal data, so that log needs its own access control and retention period.

If you would rather not build the gateway and the log yourself, our private AI/ML service includes access control and logging of queries and answers. Tell us which teams and applications call your models, and we reply within one business day.

Model files and remote code

A model file can contain code that runs when the file is loaded. Hugging Face warns: “There are dangerous arbitrary code execution attacks that can be perpetrated when you load a pickle file.” Pickle is the format of checkpoints such as pytorch_model.bin, and Hugging Face’s scan of pickle imports on the Hub “is not 100% foolproof”. Safetensors was built against this: its README asks “Can I use a file randomly downloaded and expect not to run arbitrary code?”, marks safetensors safe and pickle “Unsafe, runs arbitrary code”, and defines the file as a header length, a JSON header and a byte buffer.

For pickle checkpoints, run PyTorch 2.6 or later. Since 2.6, torch.load uses weights_only=True unless a pickle module is passed, limiting the unpickler to what state dicts of plain tensors and some primitive types need, and 2.6.0 fixed CVE-2025-32434, a remote code execution in torch.load with weights_only=True in 2.5.1 and earlier.

In Transformers, trust_remote_code=True runs modelling code that comes with the model repository rather than the library; the documentation advises loading from a specific revision, a commit hash, “to avoid loading model code that may have changed”. vLLM leaves --trust-remote-code off by default and takes --revision and --code-revision for the same pinning. Its advisory GHSA-h3rc-6mm3-gc2m of 6 October 2026 adds that, with --trust-remote-code on, a multimodal request could set the processor code revision itself through mm_processor_kwargs.code_revision and so load code from a revision an attacker controls, such as a pull request on the model’s Hugging Face repository; releases from 0.7.3 before 0.31.0 are affected, and 0.31.0 fixes it. vLLM published about thirty security advisories in September and October 2026, several of them for denial of service without authentication, so run 0.31.0 or later, preferably the current release, as of November 2026 0.33.0 of 2 November 2026, whose CUDA 13.0 builds need an R580 or newer driver; since 0.32.0 vLLM turns cross-origin browser access off unless --allowed-origins is set. vLLM also assumes its cache directories are “private and trusted”, so whoever can write to them may make vLLM run code: only the service account should write to VLLM_CACHE_ROOT and the model store.

Several tenants on one GPU: MIG, time-slicing and virtual machines

MIG gives each instance “separate and isolated paths through the entire memory system”, and for multi-tenant use NVIDIA states that “MIG ensures one client cannot impact the work or scheduling of other clients”. Time-slicing offers none of that, since “there is no memory or fault-isolation between replicas”: treat time-sliced pods as one trust domain. Our Kubernetes guide compares the methods and the MIG-capable cards.

MIG partitions the GPU, not the driver: every container still uses the host driver, where a local flaw such as CVE-2026-47489 sits. Tenants that must not trust each other belong in separate virtual machines with a passthrough GPU or a vGPU, behind a guest kernel and a hypervisor. With vLLM’s prefix caching, which is on by default, differences in time to first token can reveal whether another user’s cached prompt began the same way. Its documentation recommends setting cache_salt on every request, “using a secret scoped to the tenant boundary you want to enforce”.

Confidential computing protects weights and data in use from the host itself. NVIDIA lists it as supported on the H200 NVL (product page) and the RTX PRO 6000 Server Edition (datasheet, December 2025). The GPU then works with a confidential virtual machine on a CPU with AMD SEV-SNP or Intel TDX, and in NVIDIA’s reference architecture of June 2026 model keys are released only against valid attestation evidence. That document leaves “vulnerable inference code inside the CVM”, side channels and physical attack outside its threat model, and refers to NVIDIA’s compatibility matrix, searchable by GPU, VBIOS and driver, for supported combinations.

Network segments and egress

Four segments cover most GPU servers: management for BMCs and hypervisors, storage for models, datasets and backups, a cluster segment for traffic between nodes, and a service segment where users reach the gateway and nothing else. Only a hardened jump host should reach the BMCs. Keep GPU metrics exporters on an internal segment as well. NVIDIA scores CVE-2026-47483, published on 28 July 2026, at 8.2: unauthenticated requests to the /debug/pprof endpoints of DCGM Exporter up to 4.8.2 might lead to denial of service and information disclosure, and the record also lists DCGM itself up to 4.5.2. Run later releases of both and let only Prometheus reach port 9400; our guide to GPU server monitoring covers the exporter.

With its weights local, an inference server needs no internet access. Download models through one controlled path into the model store, serve them from a local path, which vLLM’s model setting accepts, and set HF_HUB_OFFLINE=1, with which Hugging Face’s client library, used by vLLM, makes no HTTP calls to the Hub and reads only cached files. For multimodal models, vLLM’s --allowed-media-domains limits the hosts it fetches media from, against server-side request forgery, and its documentation adds VLLM_MEDIA_URL_ALLOW_REDIRECTS=0 so that redirects cannot get around that list. For backups of the model store, see backing up an AI server.

If you want the same measures across the whole network, not only on the GPU servers, our engineering partner Vixen.UNO sets up network segmentation, zero-trust access with multi-factor authentication and privileged access management, EDR/XDR on servers and event centralisation in a SIEM as part of Cyber Resilience.

What we supply

We build AI servers to order with out-of-band management over IPMI and the BMC, install the operating system, drivers, CUDA and a container runtime on request, and supply them with manufacturer warranty on every component, on one EU contract and invoice. Our GPU range includes the H200 NVL and RTX PRO 6000 Server Edition, for which NVIDIA lists confidential computing, as well as the L40S, L4 and RTX PRO workstation cards. Under the same contract, our engineering partner Vixen.UNO delivers Cyber Resilience: a segmented network with zero-trust access, backups verified by scheduled test restores, and an incident response plan with roles, actions and deadlines.

FAQ