Visual anomaly detection on GPUs with anomalib: PatchCore, memory banks and line inference

Anomaly detection trained on good parts only

Visual anomaly detection is unsupervised defect detection: the model learns what a good part looks like from images of good parts only and flags every region that deviates. anomalib, an open-source library under Apache 2.0, implements such models, among them PatchCore, which builds its model in a single pass over the good-part images without backpropagation. A quality team with thousands of good images and a handful of defect photos can start inspection this way, and a GPU server retrains a model for each product variant while a small card at the line runs the inspection.

The difference from supervised detection decides which tool fits. A YOLO-type detector learns from boxes or masks drawn around each defect class, and Ultralytics' own guidance asks for about 1,500 images per class, as our guide to visual inspection AI without data scientists explains. That suits defects that recur and can be named. Anomaly detection suits the opposite case: rare defects, defects of a kind nobody has seen yet, and lines where scrap is too rare to collect thousands of examples. The two are often combined: anomaly detection flags the unusual parts, and a supervised model or a person classifies what was found.

The method has limits too. It flags anything different from the training images, including a new batch of material with a slightly different surface, a shifted part or a change in lighting. The training images must therefore cover the normal variation of the good parts, and the camera, lens and light must stay the same between training and production.

This article is for the IT and ML team that runs it. For model training with labelled defects, see our guide to a GPU server for training vision models; for inspection as one of several plant workloads, our article on AI servers for manufacturing.

anomalib 2.7.0: models, licence and installation

anomalib describes itself as "an anomaly detection library comprising state-of-the-art algorithms", with experiment management, hyperparameter optimisation and edge inference. The repository is licensed under Apache 2.0, and 2.7.0 is the current release on GitHub as of 25 October 2026.

anomalib GitHub repository, README and release notes of v2.7.0, and the inference reference in the anomalib documentation, read 25 October 2026.

Install with pip install anomalib or uv pip install "anomalib[cu130]". On RTX PRO Blackwell cards, which have compute capability 12.0, use the CUDA 13.0 extra: CUDA 12.8 was the first toolkit to compile for that architecture, as our guide to RTX PRO Blackwell software support explains, and our article on the "no kernel image" error on sm_120 shows what happens with older builds.

Training and testing take a few lines. The README's example builds a data module, a model and an engine, then calls engine.fit(datamodule=..., model=...); the command line does the same with anomalib train --model Patchcore and a data module. For your own images, the Folder data module takes a root directory with a folder of good images (normal_dir) and, if you have them, folders of defect images and masks. When you have no defect images yet, the custom-data guide suggests its synthetic anomaly generation to create abnormal images from normal ones for validation.

Mind one dataset licence. MVTec AD, the benchmark used in the anomalib examples and in the PatchCore paper, is licensed under CC BY-NC-SA 4.0, and MVTec states that "it is not allowed to use the dataset for commercial purposes". It is fine for learning the tool; a production model is trained on your own images.

PatchCore GPU requirements: how it uses the card

PatchCore comes from the paper "Towards Total Recall in Industrial Anomaly Detection", which reports an image-level AUROC "of up to 99.6%" on MVTec AD. It does not train a network. In anomalib's implementation, a Wide ResNet-50-2 pretrained on ImageNet extracts features from two intermediate layers, layer2 and layer3, for every patch of every good image. These patch features go into a memory bank, and a coreset step keeps a representative subset, 10 per cent by default (coreset_sampling_ratio=0.1). At inference, each patch of a new image is compared with its nearest neighbours in the memory bank (nine by default); a patch far from every good patch scores high, and the scores form the anomaly map that shows where the defect is.

Three details follow from the code. The model "requires no optimization/backpropagation"; the trainer runs max_epochs=1, a single pass through the training data. The default trainer settings carry the note "only single gpu supported", so a PatchCore job uses one card, and a server with several cards runs several jobs, one per product variant. And the nearest-neighbour search is a brute-force matrix product, processed in chunks of 1,024 query patches, which GPUs handle well.

The default image size is 256 × 256 pixels, normalised with ImageNet mean and standard deviation. The backbone has 68.9 million parameters in torchvision's listing, a small model by current standards: GPU memory goes to the memory bank and the search, not to the network.

Memory bank sizes by our arithmetic

anomalib does not document how much memory a memory bank takes, so here is the arithmetic for its defaults. In torchvision's ResNet code, layer2 of a ResNet-50 has stride 8 and 512 output channels and layer3 has 1,024; the Wide ResNet-50-2 widens only the inner bottleneck channels, so the outputs stay the same. anomalib upsamples layer3 to the size of layer2 and concatenates both, which gives a 1,536-value feature for each of 32 × 32 = 1,024 patches of a 256 × 256 image. In 32-bit floats that is 6 MiB per training image before subsampling.

Our arithmetic from anomalib's PatchCore defaults (wide_resnet50_2, layer2 and layer3, 256 × 256 images, coreset ratio 0.1) and torchvision's ResNet code; 32-bit floats assumed, since the code does not fix the data type. Not a measurement. At 512 × 512 pixels every figure is four times larger.

Three conclusions follow. The memory bank that the line needs is small: even 5,000 good images leave a bank under 3 GiB, which fits a 24 GB card with room for the backbone and the batch. Training needs more, because all features are collected before the coreset step; we found no statement in the anomalib documentation on whether they are held in GPU or host memory, so plan as if they sit on the GPU and test with your image count. And the coreset step grows faster than the image count. Greedy k-center selection, the method anomalib names, compares every collected feature with each newly chosen one, so its work grows with the number of features times the size of the coreset: by our arithmetic, ten times the images with the same ratio means about a hundred times the selection work. A team that trains on thousands of images per variant either lowers the ratio, reduces the image size, or uses a card with more memory.

The search work at the line grows linearly with the memory bank. A model trained on 5,000 images needs 25 times the multiply-adds per test image of one trained on 200, so more training images make the inspection slower. Measure the time per image on the card that will sit at the line, with the image size and bank you plan to deploy; anomalib's documentation gives no rate for these cards.

We supply the cards for the line and the servers for retraining; send us your image size, the number of good images per variant and the cycle time at the line, and our team replies with a card and server proposal.

Training centrally, inspecting at the line

A common layout separates the two jobs. The central GPU server holds the image archive, trains and retrains one model per product variant, and evaluates it against a validation set; the inspection station at the line runs the exported model on each image within the cycle time.

NVIDIA product pages and datasheets as collected in our guides to the L4 and 70 W cards, and our canonical card facts, read September to October 2026. Which card fits an industrial PC depends on the PC maker's list.

The L4 is a passive card and NVIDIA's product brief requires system airflow, so it belongs in a server or edge chassis that lists it; an industrial PC with its own fans takes an actively cooled card such as the RTX PRO 4000 SFF. Our comparison of RTX PRO 4000 SFF, RTX PRO 2000 and L4 covers the three 70 W-class cards.

For the central server, PatchCore's single-GPU training favours several cards over one large one: a server with several RTX PRO 4500 Server Edition cards, or RTX PRO 6000 Server Edition cards split into MIG instances, runs one variant per card or per instance in parallel. Two 16 GB MIG instances on an RTX PRO 4500 Server Edition hold a 1,000-image PatchCore job each by the arithmetic above, if the features stay under that size; check the peak with nvidia-smi during the first runs. Larger image counts, other anomalib models that do train with backpropagation, and a shared vision training workload are reasons to choose RTX PRO 6000 Server Edition cards. Our MIG profile guide lists the instance sizes.

Export, inference and checkpoint security

anomalib exports to ONNX, OpenVINO IR and PyTorch. Its own inferencers are a TorchInferencer, which runs on cuda, and an OpenVINOInferencer for OpenVINO, Intel's toolkit, which targets CPU, GPU and NPU devices. On an NVIDIA card at the line, the PyTorch route through Engine.predict() runs directly; an ONNX export can be served by NVIDIA-side runtimes, which our guide to serving vision models with Triton, ONNX and TensorRT covers. Test the exported model against the PyTorch one on the same images before it goes to the line, since a conversion can change scores and with them the threshold.

Two changes in 2.7.0 matter for operations. The legacy ONNX exporter (dynamo=False) is removed; the default dynamo exporter needs the openvino extra for onnxscript. And checkpoint loading now defaults to weights_only=True, so older checkpoints that pickled live objects fail to load until they are migrated once or re-saved; plan a retraining or re-export when you upgrade. The documentation also warns that TorchInferencer loads models with Python's pickle module, which can run code embedded in a checkpoint, asks to "Load models only from trusted sources" and requires TRUST_REMOTE_CODE=1. Treat model files moved from the training server to the line like software releases: signed or at least checksummed, from one controlled store. Our guide to securing a GPU server covers model files in general.

Thresholds, variants and retraining

A score is not a decision. Each model needs a threshold above which a part is rejected, set on a validation set that contains good parts and, where available, real defects. Without real defects, synthetic anomalies give a first estimate, and the first weeks at the line, with a person reviewing every rejected part, give the real one. Our article on AOI false calls with AI review in SMT production shows the same review loop for automated optical inspection.

Plan one model per product variant and camera position, because the good-part appearance differs between them. A plant with 40 variants and two camera positions trains 80 models; with PatchCore's single pass each job is short, and the central server runs them as a queue, one per card or MIG instance. Retrain when the material, the supplier, the light or the camera changes, and keep each model with its training image list, its threshold and the anomalib version, so that a rejection can be traced to the model that made it.

The newer models in 2.7.0 address the shortage of images from the other side. FoundAD is described as few-shot anomaly detection and RAD as retrieval-based; WinCLIP comes with the clip extra. They are worth a test for variants with few good images, against PatchCore on the same validation set.

What we supply

You can buy the cards and servers for this from us: our team sizes the configuration to your workload and budget and replies within one business day. We supply the L4, RTX PRO 4000 SFF and RTX PRO 2000 for the line and build retraining servers to order with RTX PRO 4500 Server Edition or RTX PRO 6000 Server Edition cards, assembled and tested before delivery, on one EU contract and invoice, with manufacturer warranty on every component handled through us; quantities and lead time are confirmed in writing with the quote. Our Private AI/ML service, with our engineering partner Vixen.UNO, builds the MLOps platform on Kubernetes for training and model releases; the camera, lighting and the integration into the line controller stay with your team or your machine vision integrator.

FAQ