Securing a GPU server: the BMC, firmware, NVIDIA drivers, the Container Toolkit, inference APIs, model files and shared GPUs

The attack surface, layer by layer

A GPU server adds layers to the usual checklist, and two of them sit below the operating system: CISA describes baseboard management controllers as components that “operate separately from the operating system (OS) and firmware to allow for remote management and control, even when the system is shut down”. Versions and dates are as of September 2026.

Our summary of the sections below; sources: CISA, AMI, NVIDIA, vLLM, Ollama, PyTorch and Hugging Face documentation and CVE records, read in September 2026.

BMC and firmware

In a June 2025 technical blog, NVIDIA warns that compromised BMCs “become a platform for persistence across every system they control”. Its first recommendations: “Place BMC interfaces on isolated management networks and never expose them to the Internet”, and “Work with your vendors to ensure BMC firmware is updated and CVEs are tracked”. It also advises treating BMC events as part of logging and detection.

Isolation does not replace patching. CVE-2024-54085, in AMI’s MegaRAC SPx BMC firmware, lets an attacker, in the record’s words, “bypass authentication remotely through the Redfish Host Interface”. AMI scores it 10.0 under CVSS 4.0, and its advisory of 11 March 2025 names fixed SPx versions; CISA added it to its Known Exploited Vulnerabilities Catalog on 25 June 2025, “based on evidence of active exploitation”. Ask your server maker which BMC release contains the fix. IPMI brings an older risk: the IPMI 2.0 specification itself lets a remote attacker obtain a password hash from the BMC for offline guessing (CVE-2013-4786). Where Redfish, which runs over HTTPS, covers your needs, we would switch IPMI over LAN off; every BMC needs a long, unique password.

The same AMI advisory also lists CVE-2024-54084, a BIOS flaw in its Aptio V firmware. GPUs carry firmware too: in NVIDIA’s H200 NVL product brief, the card’s CEC device “authenticates the contents of the GPU firmware ROM before permitting the GPU to boot from its ROM”, with rollback protection and key revocation, and the L40S product page and RTX PRO 6000 Server Edition datasheet list secure boot. Record each card’s VBIOS Version, which nvidia-smi -q prints, and take GPU firmware only from the server maker or NVIDIA.

Driver fixes arrive per branch

NVIDIA publishes security bulletins on its Product Security page, with email notifications for GPU products, and in its product-security repository on GitHub as Markdown, machine-readable CSAF and CVE records. The page announces that from 1 October 2026 NVIDIA PSIRT “will only publish security bulletins on GitHub in Markdown, CSAF, and CVE formats”, and that all bulletins stay on the Product Security website in parallel; subscribe to both.

NVIDIA’s lifecycle table gives a production branch “Quarterly (or as-needed) bug and security releases for 1 year”, and a long term support branch the same for 3 years. CVE-2026-24187 shows the result: a use-after-free in the Linux display driver, published on 26 May 2026 and scored 8.8 by NVIDIA, is fixed in 595.71.05 on R595, 580.159.03 on R580 and 535.309.01 on R535, a branch whose end of life NVIDIA’s table gives as June 2026. A server takes the security release of its own supported branch; our driver guide shows how to pin one. The record also lists vGPU guest drivers and the vGPU Manager up to vGPU 20.0, 19.4 and 16.13, so the vGPU Manager on the host and the driver in each guest both need the fix. The attack vector is local, and by our reading that can include code in a container given a GPU, because GPU containers use the host’s driver.

The Container Toolkit and the GPU Operator

The NVIDIA Container Toolkit runs on the host and sets up each GPU container as it starts; on Kubernetes, the GPU Operator deploys it by default. Four flaws, two scored 9.0 by NVIDIA, show why its version matters.

CVE records published by NVIDIA (cve.org and NVD), NVIDIA’s GitHub advisory for CVE-2024-0132, and the release notes of the Container Toolkit (18 September 2026) and GPU Operator (23 September 2026). 1.17.8 is the first Toolkit release outside the records’ affected range, and GPU Operator 25.3.1 ships it; the Operator’s notes first name CVE-2025-23266 and CVE-2025-23267 in 25.3.2.

CDI protected against CVE-2024-0132 and CVE-2025-23359 but not against the hook flaws, and three of the four descriptions name a crafted image as the vehicle. Pull images only from a registry you control, pin them by digest, and limit who may start GPU containers; on Kubernetes, that is anyone allowed to create pods on GPU nodes. Check one default too: the device plugin’s README, which assumes the NVIDIA runtime as the node’s default, warns that a pod requesting no GPU gets “all the GPUs on the machine”. Since v25.10.0 the Operator injects GPUs through CDI by default and no longer makes the nvidia runtime class the default handler, and its 26.7 releases include Toolkit 1.20. An upgrade through OLM on OpenShift leaves cdi.enabled as it was, and NVIDIA’s release notes say that GPU management containers taking GPUs through NVIDIA_VISIBLE_DEVICES instead of the device plugin must set runtimeClassName: nvidia: allow that class only for trusted pods.

Inference APIs and notebooks

vLLM listens on all interfaces unless --host is set, and its security documentation for release 0.30.0 says --api-key protects only endpoints under the /v1, /v2, /inference and /cohere prefixes. /invocations serves the same inference functions without a key, and operational endpoints such as /pause need no token. The documentation’s answer is a reverse proxy that allowlists “only the endpoints you want to expose to end users”, and a firewall that admits only the API port; the ports for torch.distributed and KV cache transfer belong to trusted hosts, and traffic between nodes is “insecure by default”. Never set VLLM_SERVER_DEV_MODE=1 in production: it adds /collective_rpc, which the documentation calls “extremely dangerous”.

Ollama starts safer: it “binds 127.0.0.1 port 11434 by default”, and OLLAMA_HOST changes that. Its official container image, however, sets OLLAMA_HOST=0.0.0.0:11434, and by default Docker publishes a port given as -p 11434:11434, as in Ollama’s Docker instructions, on all host addresses. Its local API “does not require authentication”, so such a container, or a server opened with that setting, as one of NVIDIA’s DGX Spark playbooks does for remote access, answers anyone who can reach the port. Jupyter Server turns token authentication on by default, and its documentation states the stakes: “access to the Jupyter Server means access to running arbitrary code”. Our engine comparison covers the listening defaults of three more engines.

For all of them: bind the service to localhost or an internal address and put a TLS gateway in front that gives each team or application its own key, allowlists routes, limits rates and logs who asked what. Prompts and answers can contain personal data, so that log needs its own access control and retention period.

Model files and remote code

A model file can be a program. Hugging Face warns: “There are dangerous arbitrary code execution attacks that can be perpetrated when you load a pickle file.” Pickle is the format of checkpoints such as pytorch_model.bin, and Hugging Face’s scan of pickle imports on the Hub “is not 100% foolproof”. Safetensors was built against this: its README asks “Can I use a file randomly downloaded and expect not to run arbitrary code?”, marks safetensors safe and pickle “Unsafe, runs arbitrary code”, and defines the file as a header length, a JSON header and a byte buffer.

For pickle checkpoints, run PyTorch 2.6 or later. Since 2.6, torch.load uses weights_only=True unless a pickle module is passed, limiting the unpickler to what state dicts of plain tensors and some primitive types need, and 2.6.0 fixed CVE-2025-32434, a remote code execution in torch.load with weights_only=True in 2.5.1 and earlier.

In Transformers, trust_remote_code=True runs modelling code that comes with the model repository rather than the library; the documentation advises loading from a specific revision, a commit hash, “to avoid loading model code that may have changed”. vLLM leaves --trust-remote-code off by default and takes --revision and --code-revision for the same pinning. It also assumes its cache directories are “private and trusted”, so whoever can write to them may make vLLM run code: only the service account should write to VLLM_CACHE_ROOT and the model store.

Several tenants on one GPU

MIG gives each instance “separate and isolated paths through the entire memory system”, and for multi-tenant use NVIDIA states that “MIG ensures one client cannot impact the work or scheduling of other clients”. Time-slicing offers none of that, since “there is no memory or fault-isolation between replicas”: treat time-sliced pods as one trust domain. Our Kubernetes guide compares the methods and the MIG-capable cards.

MIG partitions the GPU, not the driver: every container still uses the host driver, where a local flaw such as CVE-2026-24187 sits. Tenants that must not trust each other belong in separate virtual machines with a passthrough GPU or a vGPU, behind a guest kernel and a hypervisor. vLLM adds a subtler channel: with prefix caching, on by default, differences in time to first token can reveal whether another user’s cached prompt began the same way. Its documentation recommends setting cache_salt on every request, “using a secret scoped to the tenant boundary you want to enforce”.

Confidential computing protects weights and data in use from the host itself. NVIDIA lists it as supported on the H200 NVL (product page) and the RTX PRO 6000 Server Edition (datasheet, December 2025). The GPU then works with a confidential virtual machine on a CPU with AMD SEV-SNP or Intel TDX, and in NVIDIA’s reference architecture of June 2026 model keys are released only against valid attestation evidence. That document leaves “vulnerable inference code inside the CVM”, side channels and physical attack outside its threat model, and refers to NVIDIA’s compatibility matrix, searchable by GPU, VBIOS and driver, for supported combinations.

Network segments and egress

Four segments cover most GPU servers: management for BMCs and hypervisors, storage for models, datasets and backups, a cluster segment for traffic between nodes, and a service segment where users reach the gateway and nothing else. Only a hardened jump host should reach the BMCs.

With its weights local, an inference server needs no internet access. Download models through one controlled path into the model store, serve them from a local path, which vLLM’s model setting accepts, and set HF_HUB_OFFLINE=1, with which Hugging Face’s client library, used by vLLM, makes no HTTP calls to the Hub and reads only cached files. For multimodal models, vLLM’s --allowed-media-domains limits the hosts it fetches media from, against server-side request forgery, and its documentation adds VLLM_MEDIA_URL_ALLOW_REDIRECTS=0 so that redirects cannot get around that list. For backups of the model store, see backing up an AI server.

Network segmentation, zero-trust access with multi-factor authentication and privileged access management, EDR/XDR on servers and event centralisation in a SIEM are part of the Cyber Resilience service described below.

What we supply

Eurokommerz builds AI servers to order with out-of-band management over IPMI and the BMC, installs the operating system, drivers, CUDA and a container runtime on request, and supplies them with manufacturer warranty on every component, on one EU contract and invoice. Our GPU range includes the H200 NVL and RTX PRO 6000 Server Edition, for which NVIDIA lists confidential computing, as well as the L40S, L4 and RTX PRO workstation cards. Under the same contract, our engineering partner Vixen.UNO delivers Cyber Resilience: a segmented network with zero-trust access, backups verified by scheduled test restores, and an incident response plan with roles, actions and deadlines.

FAQ