AI server for e-commerce: product texts in many languages, search, shop chatbot, images

What an AI server does for an online shop

An AI server for e-commerce with NVIDIA RTX PRO 6000 Blackwell Server Edition cards runs the language, search and image models of an online shop on hardware the company controls. It writes and translates product texts, turns the catalogue into vectors for search and recommendations, answers customers in a shop chatbot and edits product images. For a mid-sized shop, two cards carry the chatbot and the search models, and four cards add overnight product texts for tens of thousands of items in several languages.

The reader we have in mind runs a shop with its own catalogue of a few thousand to a few hundred thousand products, sold in several European countries and described in each of their languages. Every figure below is either a published measurement, with its source, or our estimate, labelled as such.

Four jobs with four different loads

The four jobs share a server but stress it in different ways, and that difference decides the configuration more than the number of products does.

Our summary; models as discussed in the sections below.

Product texts can wait until the night, so throughput matters and latency does not. Search must answer within a page load, so each query needs a fast embedding and, if you rerank, a fast reranker. The chatbot holds a working memory on the GPU for every conversation waiting for an answer. Image editing spends several seconds of GPU time on every picture. A shop can start with one job and add the others on the same server if the cards are planned for it. Our guides for creative agencies and media companies apply the same approach to their work.

Product texts in many European languages

Most shops write a product text from structured data: attributes, dimensions, materials, the supplier's description. A language model turns that into a readable text in the shop's tone, in every language the shop sells in. The language list on the model card is the first filter. The Mistral Small 3.1 card names 25 languages, German, Polish and Romanian among them, and Qwen3's launch post lists 119. Llama 3.3's card names eight languages, German among them but not Polish or Romanian, and gpt-oss has no language list on its card. EuroLLM covers all 24 official EU languages under Apache 2.0. Our comparison of open LLMs for European languages has the details and the licence of each.

There are two ways to cover many languages. The model can write each language directly from the attributes, or it can write one master text that a translation model then renders into the other languages. The second way keeps the facts identical across languages and lets an editor approve one text instead of eight. Our guide to on-premise machine translation sizes translation models from words per day: by its estimate, one 48 GB card with a 9B to 12B translation LLM handles about one million words a day. It also notes that the NLLB-200 cards call it a research model, not released for production deployment.

The throughput figure that comes closest to this job is StorageReview's test of four RTX PRO 6000 Server Edition cards in an HPE ProLiant Compute DL380a Gen12 with vLLM. At batch 32, gpt-oss-120b in NVFP4 at TP=4 gave each of the 32 users 47.54 tokens per second, about 1,520 output tokens per second together by our arithmetic. StorageReview also reports 4,016 tokens per second in total at batch 64, without stating whether that total includes the prompt tokens, so we plan with the lower figure. A dense model is much slower here: Llama 2 70B gave 8.00 tokens per second per user at batch 32 in the same review.

We plan with 2 output tokens per word, as our translation guide does; a text of 120 words is then about 240 tokens. At 1,520 tokens per second that is about 6.3 texts a second, or 22,800 an hour. A worked example: 20,000 new or changed products in six languages make 120,000 texts and 28.8 million output tokens, which take about 5.3 hours on four cards by our estimate, inside one night. The prompt with the attributes comes on top and the review's prompt lengths are not yours, so run a test with a few hundred of your own products before you size the server. Our guide to batch LLM inference on-premise explains how to queue such a run. We build servers for this kind of nightly job; send us your product count, languages and the text length you need and our team replies with a configuration.

Search and recommendations with embeddings

Classic shop search matches words. Embedding search turns every product text and every query into a vector and finds the nearest vectors, so a query for "waterproof jacket for cycling" can find a product described as "rain shell for bike commuters". The same vectors drive recommendations: the nearest neighbours of a product are its "similar items", and they can be computed overnight and written into the shop database, so no GPU sits in the path of a page request.

The models are small. Qwen3-Embedding comes in 0.6B, 4B and 8B under Apache 2.0, with more than 100 languages and inputs of up to 32K tokens, and BAAI's bge-m3, under MIT, has 568M parameters and 8,192 tokens of input. Qwen3-Embedding-0.6B takes about 1.2 GB in BF16. Qwen3-VL-Embedding, published in January 2026 in 2B and 8B, also embeds images, which suits "visually similar" recommendations from product photos. Our guide to embedding and reranker servers for RAG lists them with their licences.

NVIDIA NeMo Retriever embedding NIM (version 1.14.0, updated 30 September 2026) and reranking NIM (updated 17 August 2026) performance pages, FP8, as quoted in our embedding and reranker guide; reranking at three requests in parallel. Shop examples our arithmetic.

If each product is embedded in six languages, the 200,000 products become 1.2 million texts and take about 2.3 hours on one L40S by the same arithmetic, which a nightly job absorbs. A million 1,024-dimensional vectors in float32 take 4.10 GB before any index, as our vector database comparison shows, so even a large multilingual catalogue fits in the memory of an ordinary database server.

The reranker is the part that grows with traffic. It scores every retrieved product against the query, so its load is queries per second times candidates per query. A search box that receives dozens of queries a second at peak needs several cards to rerank every query; reranking only the chatbot's retrieval, or only the first page of results, keeps it on one card or one MIG instance. Measure your peak queries per second from the shop's logs before you decide.

The shop chatbot

A shop chatbot answers questions about products, delivery and returns from your catalogue and help pages, and looks up an order through your shop's API. Our guide to a server for a customer service chatbot sizes it in detail; its figures carry over. By our users-per-card rule, gpt-oss-120b, a mixture-of-experts model under Apache 2.0 whose checkpoint of 65.3 GB fits one 96 GB card, holds 39 chats in flight on one Server Edition card at a declared context of 16,384 tokens and 187 on two cards as a tensor-parallel pair. With that guide's example values, one customer message a minute and 20 seconds per answer, a third of the open chats are in flight, so a pair serves about 560 open chats.

The rule takes 90 per cent of the 95.6 GiB the driver reports on a 96 GB card, subtracts 3 GiB per card and the weights, and divides the rest by the cache per chat. For the pair: 2 × 83.04 = 166.1 GiB, less 60.8 GiB of weights, leaves 105.3 GiB, and 105.3 ÷ 0.5625 GiB per chat gives 187. Our how many users per RTX PRO 6000 article explains the rule.

Two points are specific to a shop. Stock, prices and order status change by the minute, so the bot reads them through tool calls to the shop's API at answer time rather than from an index. And a bot on the storefront is a public service: if it must keep answering while a server is updated or a card fails, plan two servers that each carry the peak, as our guide to LLM high availability with two GPU nodes describes.

Product image editing and its licences

Image models change backgrounds, place a product in a scene or produce colour variants. For a shop, the licence on the model card comes before the GPU, because product images are commercial use.

Hugging Face model cards of black-forest-labs and Qwen, read 24 October 2026; Kontext memory from NVIDIA as quoted in our FLUX guide; Qwen-Image-Edit weights our arithmetic at 2 bytes per parameter, without the text encoder.

The FLUX.2 [klein] 4B card states "multi-reference editing capabilities" and "Open weights available for commercial use". The Qwen-Image-Edit card describes semantic and appearance edits and "bilingual (Chinese and English) text editing", so text inside an image in German or Polish is outside what the card claims. The FLUX.1 Kontext [dev] card says the model falls under the FLUX.1 [dev] Non-Commercial License, while also saying that generated outputs can be used for commercial purposes; whether your use fits is a question for your legal department. Our FLUX GPU requirements guide covers the commercial licences Black Forest Labs sells.

For speed, NVIDIA lists one RTX PRO 6000 Server Edition at 0.20 FLUX images per second in FP4 at batch 1, about 720 an hour, without naming the variant, resolution or step count. One step of FLUX.1 Kontext [dev] takes 254 ms in FP4 on the RTX PRO 6000 by NVIDIA's figures, so 28 steps spend about 7.1 seconds in the transformer. A seasonal refresh of 10,000 product images would take about 14 hours on one card at NVIDIA's rate and about 3.5 hours on four, by our arithmetic. FLUX.2 [klein] 4B, at about 13 GB, fits a 24 GB MIG instance, so one card split into four instances can run four editing queues at once.

Configuration options by card count

All options use NVIDIA RTX PRO 6000 Blackwell Server Edition cards and follow the reference configurations on our AI servers page and NVIDIA's guide for certified systems: two processors, at least six cores per GPU and system memory of twice the GPU memory.

Reference configurations on our AI servers page; chats by our users-per-card rule at 16K with the example values of our chatbot guide; power by our estimate (cards at 600 W, processors, plus 15 per cent for the rest of the server).

Two cards. The entry server, two RTX PRO 6000 Server Edition with two AMD EPYC processors, 384 GB of DDR5 ECC and two NVMe drives, suits a shop that starts with search and a chatbot for a small team of service agents. MIG splits the second card into four isolated 24 GB instances: the embedding model, the reranker and two image-editing queues. Product texts run at night on the chatbot card, more slowly than in StorageReview's four-card test. Our page on the two-card server has the details.

Four cards. The inference node with 768 GB and four NVMe drives is the configuration we would size first for a shop that runs all four jobs. The chatbot pair serves about 560 open chats; at night, when chat traffic is low in the shop's markets, all four cards can run the product-text batch at TP=4, the layout StorageReview measured. Our page on the four-card server covers the layouts.

Two four-card servers or eight cards. A shop that sells across time zones has no quiet night, and a storefront chatbot should survive a fault. Two four-card servers give both: each carries the chat peak alone, and while both run, one of them works through the text and image batches. Eight cards in one server give the same capacity in one chassis, at about 6.3 kW and without failover.

The RTX PRO 6000 Max-Q is the workstation card, a 300 W blower card; in a rack it belongs only in a chassis whose maker lists it, and for continuous operation we recommend the Server Edition.

Notes for the IT team

1. Integration. vLLM serves an OpenAI-compatible API for the language models and has pooling endpoints for embeddings and scoring, so the shop backend calls one interface. Put an LLM gateway with keys and a query log in front of it.
2. Network. vLLM's --api-key protects only paths under /v1, /v2, /inference and /cohere, and its security page lists /pooling, /score and /rerank among the routes it does not cover. Keep the engines on an internal network behind the gateway. Keep the NVIDIA Container Toolkit at 1.17.8 or later and the driver at a release NVIDIA lists as fixed in its bulletin of 30 September 2026.
3. Engines. Text Embeddings Inference 1.9 runs on RTX PRO Blackwell cards only from an image it marks experimental; the Qwen3 embedding and reranker cards require vLLM 0.8.5 or later, and on a company network run 0.31.0 or later because earlier releases have published security flaws. NVIDIA's NIM containers need NVIDIA AI Enterprise in production.
4. Software on delivery. The server arrives on request with the operating system, the NVIDIA driver, CUDA and a container runtime; the models and their pipelines go on top.
5. Data and law. Chat logs hold customer data, so set retention and access rules and check whether a data protection impact assessment is needed. Since 2 August 2026, Article 50 of the AI Act requires that people are told they are interacting with an AI system unless that is obvious, and providers of generative systems mark generated images in a machine-readable format, with an exception for assistive standard editing. A shop that assembles its own image pipeline can be its provider. Our article on the EU AI Act gives general information; national practice may differ and your legal department decides.

Power and room

By our estimate, the two-card server draws about 1.8 kW at the wall with two EPYC 9135 processors and fits one 230 V 16 A feed, which carries 2.94 kW at 80 per cent. The four-card server draws about 3.4 kW with the cards at 600 W, or about 2.7 kW at the 450 W cap, and needs 32 A or three-phase. Eight cards draw about 6.3 kW by our estimate; Dell's datasheet gives about 8 kW per eight-card node as an indicative figure, and feeds should be planned on the maker's figure. Our guide to GPU server power on 16 A, 32 A and three-phase feeds works through each case.

What we supply

You can buy this system from us: our team sizes the configuration to your workload and budget and replies within one business day. We build the servers to order with two, four or eight RTX PRO 6000 Server Edition cards, assembled and tested before delivery, delivered anywhere in the EU on one EU contract and invoice, with manufacturer warranty on every component handled through us; quantities and lead time are confirmed in writing with the quote. Our Private AI/ML service, with engineering by our partner Vixen.UNO, deploys the language, embedding and image models and the RAG pipeline behind them. We supply the server and set up the model and retrieval behind the bot; a simple website chatbot is not what we do, so the chat widget in your storefront and its connection to the shop platform stay with your team or your shop platform provider. Describe your catalogue, languages and traffic, and we propose the server for them.

FAQ