Nextcloud Assistant with a local LLM: requirements, GPU per feature and users per card

Nextcloud AI on premises: the features on your own hardware

Nextcloud Assistant can run with a local LLM entirely on your own servers: Nextcloud's own apps run a language model, answer questions about your files and translate text on premises, and for larger models or more users the Assistant connects to an OpenAI-compatible model server such as vLLM on a GPU server. The text app needs an NVIDIA GPU with at least 8 GB of memory, and Nextcloud states that this app and Context Chat cannot be scaled yet, so a single instance serves roughly 1,000 requests an hour. Above that, a separate GPU server with an NVIDIA RTX PRO 4500 Blackwell Server Edition or RTX PRO 6000 Blackwell Server Edition carries the load and can serve other departments too.

Many organisations in Europe run Nextcloud to keep their files on their own servers. The Assistant brings summaries, drafting help, chat and questions over documents into the same place, which raises one practical question for the management: which hardware keeps those AI features on premises as well. This guide answers it first in plain terms, then in a technical part for the IT team. Every Nextcloud fact is from the "latest" Nextcloud administration manual and Nextcloud's GitHub pages, read on 25 October 2026; the manual changes with each release.

What the management needs to decide

The Assistant itself is only the window in Nextcloud where staff type a request. Behind it, a "provider" does the work, and there are two kinds:

• Nextcloud's own local apps. They run on a machine you control, use small open models and need little set-up. Their limit is capacity: for the text app llm2 and for Context Chat, Nextcloud states that they cannot be scaled yet.
• A connection to a model server. Nextcloud's "OpenAI and LocalAI integration" app sends the requests to any service that speaks the same interface as OpenAI's. That service can be a GPU server in your own data centre running vLLM, a widely used open-source engine. Nothing leaves your network in this set-up either.

Three questions decide the hardware: how many staff will use the Assistant at the busiest hour, whether they need answers from their own files (Context Chat), and whether meeting recordings should be transcribed. A pilot with one department answers the first; the other two follow from what you plan to offer.

Nextcloud administration manual, AI section (llm2, Context Chat), read 25 October 2026; card capacity by our users-per-card rule as explained below.

The second and third set-ups also suit organisations whose other systems will use AI later: one GPU server can serve Nextcloud, a chat front end and a coding assistant at once. Our guide to a private ChatGPT alternative describes that wider set-up.

What each Nextcloud AI app needs

Nextcloud's overview lists which app provides each feature on premises. These are the requirements its manual states for the apps that matter for hardware:

Nextcloud administration manual, pages for llm2, Context Chat, Local Whisper Speech-To-Text and Recognize, read 25 October 2026.

llm2 runs on llama.cpp and "is thus compatible with any model in gguf format". Nextcloud tests only with Llama 3.1, whose supported languages it lists as English, Portuguese, Spanish, Italian, German, French, Hindi and Thai, and names OLMo as a fully open alternative that works well only with English and cannot use tools. The app needs CUDA 12.2 or later on the host, a CPU with AVX and AVX2, and "will hog all cores by default". It is an External App (ExApp) that runs in a Docker container through AppAPI and a deploy daemon.

Context Chat indexes your files into a bundled PostgreSQL database with vector support and needs a separate text provider, llm2 or the OpenAI integration, to write the answers. Its manual warns that the initial indexing "can take a long time", that synchronous indexing "could take several days", and that it runs in the maintenance window. As a size guide it says "20 MB should be large for a CPU-backed setup and 100 MB for a GPU-backed system"; password-protected PDFs are not supported.

stt_whisper2 uses faster-whisper with Whisper large-v2 or v3 for many languages or medium.en for English. Its page says plainly "Using GPU is currently not supported", so a GPU does nothing for it.

Recognize tags faces, objects, landscapes, music genres and actions in videos. GPU processing "is supported, but not required", for NVIDIA cards with driver 450.80.02 or later, CUDA Toolkit 11.x and cuDNN 8.x. RTX PRO Blackwell cards need software built with CUDA 12.8 or later, as our article on Blackwell software support shows, so on those cards plan Recognize on the CPU unless its current release states otherwise.

Capacity: what one local app instance carries

For llm2 Nextcloud writes: "It is currently not possible to scale this app, we are working on this. Based on our calculations an instance has a rough capacity of 1000 user requests per hour", and adds that the number "is based on theory". Context Chat carries the same two statements. A thousand requests an hour is about one every 3.6 seconds.

What that means in staff depends on use, and we found no published ratio between headcount and AI requests. With the example values of our guide to sizing a private ChatGPT server by company size, 40 per cent of staff active in the busiest hour with 6 requests each, 1,000 requests an hour corresponds to about 420 employees, or about half that if the busiest minutes run at twice the hourly rate, the peak factor in the same example. Those are example values; the Assistant's own task history in a pilot replaces them.

Two settings affect what staff perceive. Nextcloud processes AI tasks through its background job system, which "only runs jobs every 5 minutes by default", and its AI-as-a-Service page warns that prompts "can have a delay of up to 5 minutes". The manual's fix is dedicated workers: run occ background-job:worker for OC\TaskProcessing\SynchronousBackgroundJob "4 or more times", for example as systemd services. Set this up before judging the GPU; otherwise the pilot mostly measures waiting time in the queue.

We supply L4, RTX PRO 4500 Server Edition and RTX PRO 6000 Server Edition cards and GPU servers built to order; tell us your number of Nextcloud users and the AI features you plan, and our team replies with a configuration for the local apps or for a vLLM server.

The vLLM route through the OpenAI integration

Nextcloud's "OpenAI and LocalAI integration" app (integration_openai) can "connect to a self-hosted LocalAI or Ollama instance" or "any service that implements an API similar enough to the OpenAI API". Its README lists what it provides per model type: text features such as "Free prompt, Chat, Chat with tools, Summarize", image generation, "transcription, subtitles and transcription with paragraph reformatting" for transcription models and text to speech. The administrator connects any number of services, sets the API key per service and configures usage quotas per service.

vLLM is not named on Nextcloud's pages. Its documentation describes an HTTP server with the OpenAI endpoints /v1/chat/completions, /v1/completions, /v1/embeddings and, for speech recognition models, /v1/audio/transcriptions, which is the interface the integration expects. The maintainers write that the app is "mainly tested with OpenAI" and that other models and APIs may not work, so run the Nextcloud features you plan against your vLLM server in a pilot before rollout. Start vLLM with --api-key and enter the key in Nextcloud; the key protects only paths under /v1, /v2, /inference and /cohere (vLLM 0.32.0 security guide), so keep the engine on the internal network.

This route has three advantages for a growing installation. vLLM batches many requests on one card, so capacity grows with GPU memory. Blackwell support is documented for vLLM, which removes the question whether an ExApp's build supports the card. And the same server answers other clients, which is how RAG on company data and Nextcloud can share one GPU budget.

Which GPU for which feature

Our users-per-card rule takes 90 per cent of the memory the driver reports and subtracts 3 GiB per card; what the weights leave holds the KV cache, the working memory of each conversation in flight. Both models below are under the Apache 2.0 licence according to their LICENSE files on Hugging Face.

Weights as the sums of the safetensors files on Hugging Face: gpt-oss-20b 12.8 GiB, gpt-oss-120b 60.8 GiB, read 25 October 2026. Cache per conversation with a 16-bit KV cache from each config.json: gpt-oss-20b 0.75 GiB per 32K, gpt-oss-120b 1.125 GiB. L4 with ECC on. Our estimates, not measurements.

The text features. For the local llm2 app, Nextcloud's 8 GB minimum makes the L4 (24 GB, 72 W, low profile) a natural fit inside an existing Nextcloud server whose maker lists the card. Its Ada architecture also avoids the Blackwell build question for llama.cpp. For the vLLM route, the RTX PRO 4500 Server Edition is the card we recommend: 32 GB, 165 W, passive and single-slot, and splittable by MIG into two isolated 16 GB halves. With gpt-oss-20b it carries 17 conversations of 32K tokens, or 68 of 8K, which covers most Assistant requests such as summaries and drafts.

Context Chat. It needs 2 GB of GPU memory for its embedding work and the text provider for the answers. On a vLLM server, one MIG half of an RTX PRO 4500 Server Edition can hold small embedding and reranker models, such as the 0.6B Qwen3 models, for a separate retrieval service, as our guide to an embedding and reranker server explains.

Speech. The local Whisper app is CPU-only. For transcription on a GPU, connect a speech server through the integration's transcription models; our guide to Whisper GPU requirements sizes it, and an RTX PRO 4500 Server Edition carries recognition and diarisation.

More users and a larger model. gpt-oss-120b, with 117B parameters against 21B for gpt-oss-20b, runs on an RTX PRO 6000 Server Edition with 19 conversations of 32K per card. The entry configuration on our AI servers page, with two such cards, two AMD EPYC processors and 384 GB of DDR5 ECC memory, runs one copy per card and draws about 1.8 kW at the wall by our estimate; with the example values of our company-size guide, 500 employees produce about 20 requests in flight at the peak, which the two copies cover with room to spare, and a second server adds redundancy.

Access rights and data protection

Context Chat answers from the files a user may read, and its manual states that "users who never had access to a particular file/folder will NOT be able to see those contents in any way". It also lists a known issue: users who had access to files or folders and lost it may "still have access to the content of those files/folders through the Context Chat app". Nextcloud says it is working on it. For an organisation that revokes access when people change roles, that matters, and your data-protection officer should review it before Context Chat is enabled for everyone.

The manual's general caveats belong in the rollout notes as well: language models "are likely to generate false information", Nextcloud's support covers implementation bugs, not model output, and output should be checked. Nextcloud's overview gives each AI provider an "Ethical AI Rating" from green to red, which helps document the choice of model.

What we supply

We supply the L4, the RTX PRO 4500 Server Edition and the RTX PRO 6000 Server Edition as cards with manufacturer warranty for existing servers, and GPU servers built to order, assembled and tested before delivery, with Linux, the NVIDIA driver, CUDA and a container runtime installed on request. Our engineering partner Vixen.UNO sets up vLLM, the models and the connection to Nextcloud within our Private AI/ML service. Quantities and lead time are confirmed in writing with the quote, on one EU contract and invoice. If you want a configuration for your installation, send us your user count and the Nextcloud features you plan to offer.

FAQ