Federated learning for hospitals: the GPU server each site needs, and the aggregator

What federated learning changes for a hospital

Federated learning lets several hospitals train one model without moving patient data: each site trains the current model on its own data, sends the resulting update to an aggregation server, and receives the combined model for the next round. Every participating hospital therefore needs a GPU server of its own, sized for the local training job, while the aggregator that combines the updates is a small machine. For 3D imaging or language models, a site node with two NVIDIA RTX PRO 6000 Blackwell Server Edition cards is the size we would start from; for 2D images, tabular data and smaller networks, four RTX PRO 4500 Server Edition cards in 2U fit.

We wrote this for the IT lead of a hospital that joins a research consortium and for the research team that plans it. The question of what one hospital needs for its own AI is covered elsewhere, in our guides to the medical imaging AI server and the private LLM server for hospitals. This page is about the multi-site pattern: the node per site, the network between sites, and what the regulations say about data that stays where it is.

How a federated round works

NVIDIA FLARE describes itself as "a domain-agnostic, open-source, extensible Python SDK" that moves existing machine learning workflows to a federated setup. It is published under Apache 2.0 and supports PyTorch, TensorFlow, scikit-learn and XGBoost. Version 2.9.0 appeared on 4 September 2026 and needs Python 3.10 or later. Flower, also under Apache 2.0, is a framework-agnostic alternative whose README names MONAI among the libraries it works with.

NVIDIA FLARE documentation, real-world deployment overview (main branch), and the NVFlare README and release notes for 2.9.0 on GitHub, read 24 October 2026.

A project starts with provisioning. nvflare provision or the FLARE Dashboard generates a startup kit for the server, for each client and for each administrator, and the kits travel to the participants over a secure channel. Each kit holds the root certificate and the participant's own certificate and key; the identity sits in the certificate's common name, and FLARE's documentation tells each site to safeguard its client key. The kit contents are signed, so a changed file is detected. Clients connect to the server, which binds to the hostname set at provisioning, so the server's port must pass every firewall on the way and that hostname must resolve at each site. Release 2.9.0 made mutual TLS the default for FLARE's internal links in Docker, Slurm and Kubernetes deployments.

The training code stays the hospital's own. FLARE wraps a PyTorch training loop as a federated job, release 2.9.0 can federate an existing Hugging Face Trainer or TRL SFTTrainer through flare.patch(trainer), and it adds a Slurm launcher, so a hospital that already runs a research cluster can schedule the FL client like any other job; our guide to a GPU server for a university research lab covers GPU scheduling in Slurm.

What each site trains, and the GPU memory it needs

The GPU node at each site has to run one local training job of the shared model, and the model sets the memory. The figures below come from the model documentation summarised in our imaging guide and from our fine-tuning sizing.

MONAI Model Zoo READMEs as cited in our medical imaging AI server guide (read 10 October 2026); 70B QLoRA from our fine-tuning sizing, computed from listed card capacity. The card column is our reading.

Language models can be trained this way as well. FLARE's release notes for 2.9.0 state that federated averaging was validated "up to a 72-billion-parameter model". A site that fine-tunes a 70B model with QLoRA needs about 42.8 GB on the card, which leaves 53 GB on a 96 GB RTX PRO 6000 for batch and sequence length; our guide to GPU memory for LoRA and QLoRA fine-tuning covers the arithmetic. System memory matters as well: MONAI's whole-body bundle reports 83 GB of system RAM on one GPU for a CacheDataset of 1,000 volumes at a cache rate of 0.4, so a site node needs RAM for the data cache on top of NVIDIA's rule of twice the GPU memory.

Site configurations by card count

The sites can run different nodes, but each must run the same training job. In federated averaging a round combines the updates of all sites, so the slowest node paces the round, and a consortium does well to agree a minimum node.

Reference configurations on our AI servers page and NVIDIA's guide for certified systems (system memory twice the GPU memory); four RTX PRO 4500 Server Edition from our buying guide for that server; power by our estimate (cards, processors and 15 per cent for the rest of the server). The FLARE pages we read (README, release notes, real-world deployment overview) state no hardware requirement for the server.

Four RTX PRO 4500 Server Edition. Four passive single-slot cards with 32 GB each fit a 2U server and one 230 V 16 A feed, which carries 2.94 kW at 80 per cent. Each card also splits into two 16 GB MIG instances, so the FL job and a local inference service can share the server with isolated memory. This suits a site in a consortium that trains 2D models or classifiers.

Two RTX PRO 6000 Server Edition. The entry server from our AI servers page, with two AMD EPYC processors, 384 GB of DDR5 ECC and two NVMe drives, holds 3D segmentation training on one card and keeps the second card for the hospital's own work, or trains on both. It is the node we would suggest as the consortium minimum for imaging and language models.

Four RTX PRO 6000 Server Edition. The inference node with 768 GB suits the coordinating site that also develops the training job, tests it on its own data before each campaign and may host the FL server in a separate VM.

The RTX PRO 6000 Max-Q is the workstation card, a 300 W blower card; in a rack it belongs only in a chassis whose maker lists it, and for continuous operation we recommend the Server Edition.

We build the same node for every site of a consortium and ship it to each hospital in the EU on one contract; send us the number of sites, the model and the data per site, and our team replies with a configuration.

The network between sites

Only model updates leave the hospital, but they are not small. A full update has one value per parameter: the H-optimus-0 pathology model with 1.1B parameters ships as a 4.54 GB file, which takes about 36 seconds each way over 1 Gbit/s and about 3.6 seconds over 10 Gbit/s, by our arithmetic. A campaign of 100 rounds moves about 0.9 TB per site for such a model. A segmentation network with 100 million parameters sends about 400 MB per update in FP32, about 3 seconds at 1 Gbit/s. A job that trains only LoRA adapters needs to exchange only the adapter weights, a small fraction of the model.

In FLARE's documentation the clients connect to the server, and it is the server's port that must not be blocked by any firewall; plan the rules on that basis and confirm them in a test run before the first campaign. Agree the hostname, the certificate authority and the firewall rules before provisioning, because the startup kits carry them. The FL client sits in a research network zone with read access to the prepared training data and no access to clinical systems beyond that; our guide to network segmentation describes the zones.

What the model updates can reveal

An update is computed from patient data, so it carries information about that data. FLARE lists "differential privacy, homomorphic encryption, private set intersection (PSI)" under privacy preservation, and adds federated authorisation and a privacy policy that each site enforces for its own client. Which of these a consortium uses is part of its protocol and its data protection impact assessment. For language models, the EMA's reflection paper on AI warns that "Large language models, often containing billions of parameters, are at particular risk of memorisation due to their size", which matters when a federation fine-tunes one on clinical text. The final model is a result to be reviewed before it leaves the consortium.

EHDS, GDPR and the hospital's role

The European Health Data Space regulation, Regulation (EU) 2025/327 of 11 February 2025, entered into force on 26 March 2025. The Commission's EHDS page gives the timeline: its rules on secondary use start to apply for most data categories, such as data from electronic health records, in March 2029, and for the remaining categories, such as genomic data, in March 2031. Secondary use is "based on a permit issued by a health data access body", data processing "can only take place in secure processing environments", and "No personal data can be downloaded from these environments". Recital 77 states that all secondary-use access "should be done through a secure processing environment", and recital 80 that bringing questions to data instead of moving those data "should be respected whenever possible".

Federated learning keeps data at the site, which is the approach recital 80 describes. Whether a given federated setup, or the participation of a hospital in one, meets the regulation's rules on permits and secure processing environments is not answered by the passages we read, and it depends on the data, the purpose and national implementation. Health data stays a special category under Article 9(1) of the GDPR, and large-scale processing of such data is among the cases in Article 35(3)(b) that require a DPIA; our guide to a DPIA for an internal LLM lists the technical inputs. The legal basis and the consortium agreement are for each hospital's data-protection officer and legal counsel. General information on EU law as of October 2026; national rules and transposition may differ.

Notes for the IT team

1. Software. FLARE 2.9.0 needs Python 3.10 or later; it runs in Docker, as Slurm jobs or on Kubernetes. Pin the FLARE version across all sites for a campaign, since every client runs the same job.
2. Keys. The startup kit holds the site's private key. Store it like any service credential and restrict who can start the client.
3. GPU sharing. If the node also serves local inference, put the FL job on its own card or MIG instance, as our guide to GPU sharing on Kubernetes with MIG explains, so a training round cannot slow clinical work.
4. Data path. The client reads a prepared, pseudonymised training set from local NVMe or the research storage, never from the production systems directly.
5. Software on delivery. On request the server arrives with the operating system, the NVIDIA driver, CUDA and a container runtime. The BMC belongs on the management network, as our guide to securing a GPU server describes.

Our Private AI/ML service, with our engineering partner Vixen.UNO, sets up the FL client, the containers and the logging on these nodes; tell us which framework the consortium uses, and we prepare the node to match it.

What we supply

You can buy this system from us: our team sizes the configuration to your workload and budget and replies within one business day. We build GPU servers to order with RTX PRO 4500 Server Edition or RTX PRO 6000 Server Edition cards, assembled and tested before delivery, delivered to every site anywhere in the EU on one EU contract and invoice, with manufacturer warranty on every component handled through us; quantities and lead time are confirmed in writing with the quote. Our Private AI/ML service deploys the software on top. The consortium agreement, the DPIA and the regulatory assessment stay with the participating hospitals.

FAQ