From Ollama or LM Studio on one PC to a shared AI server for the whole company

What changes when the whole company wants the PC experiment

A local AI that runs in Ollama or LM Studio on one PC is a single-user tool. To make it a service for 20 to 100 colleagues, the company needs a server engine that answers many questions at the same time, user logins with permissions, an update and backup routine, licence terms that cover company-wide use, and graphics cards with enough memory for every conversation in progress. The model you liked and the questions you tested carry over; the desktop software and its file formats are usually replaced.

This guide is written in two parts. The first half is for the owner or managing director who saw the experiment work and now has to decide on the next step. The second half, from the section on the engine, gives the IT team or the external IT service provider the details.

Ollama and LM Studio are programs that download an open AI model (a large language model, the kind of AI behind ChatGPT) and run it on the computer they are installed on. Open WebUI is a chat page in the browser that many firms put on top of Ollama, so it looks and works like a public chatbot.

Ollama with several users: one request at a time by default

On a PC, one person asks a question and waits for the answer. In a company, ten people ask at the same moment. Ollama's documentation states the default plainly: OLLAMA_NUM_PARALLEL, "The maximum number of parallel requests each model will process at the same time", is 1. Further requests wait in a queue of up to 512 before Ollama rejects new ones. With five colleagues writing at once, four of them wait while the first answer is written.

The setting can be raised, but each parallel slot needs its own memory for the conversation. The FAQ puts it this way: "Parallel request processing for a given model results in increasing the context size by the number of parallel requests." Every conversation keeps a working memory on the graphics card, called the KV cache, which grows with the length of the text. For gpt-oss-120b a conversation of 32,768 tokens takes about 1.125 GiB, by the calculation in our guide to a private ChatGPT server by company size. The model itself takes about 60.8 GiB.

Server engines such as vLLM are built for the shared case. They keep that memory in small blocks and add new questions to the running batch as soon as a slot frees up, so the card serves many conversations at once instead of queuing them. Our comparison of vLLM, SGLang, TensorRT-LLM, llama.cpp and Ollama explains the differences for the IT team.

Logins, permissions and the network

On a PC, whoever sits at the keyboard is the user. A company service needs to know who asks, which models and documents each person may use, and who administers it.

Ollama answers only on the PC itself by default: "Ollama binds 127.0.0.1 port 11434 by default." To reach it from other computers, the bind address is changed with OLLAMA_HOST, and Ollama's own container image already sets OLLAMA_HOST=0.0.0.0:11434, so a container started that way answers on every network address of the machine. Ollama's documentation describes no login for its API, so anyone who reaches the port can use it. The GitHub advisory GHSA-w8p2-phwr-px3r, CVE-2026-103663, published on 8 October 2026 and rated critical, describes a path traversal in the /api/pull endpoint through which an unauthenticated attacker can write a file outside the model store, leading to code execution after a restart; it was fixed in Ollama 0.35.0. The current release on 25 October 2026 was 0.40.11.

Logins belong in the chat front end. In Open WebUI the first account on a new installation becomes the administrator, new sign-ups get the default role set by DEFAULT_USER_ROLE, and its documentation recommends pending for shared instances, so that an administrator approves each account. Groups decide who may read or change a model or a document collection, and with ENABLE_OAUTH_GROUP_MANAGEMENT the groups follow the company's identity provider at each sign-in. Open WebUI's permissions are additive: a user in several groups gets the combined rights, and deny rules do not exist.

Who sees which documents is the question with legal weight, and our article on preparing company documents for an AI assistant covers it for the people who own them. Our Private AI/ML service sets up the server, the logins and the document search; tell us how many colleagues use the PC set-up today and what they use it for, and our team proposes the next configuration.

Licence terms of the desktop tools

A tool that is fine for one person's experiment may have different terms for company-wide use, so read each licence before rollout. As read on 25 October 2026:

Ollama repository on GitHub (MIT); LM Studio app terms, version of 23 August 2026; Open WebUI licence page; openai/gpt-oss-120b on Hugging Face; all read 25 October 2026.

LM Studio's terms grant a licence "solely for Your personal and / or internal business purposes", and where an employee uses it, "the corporate entity is the Customer". They forbid to "sublicense, distribute, sell, use for service bureau use, as an application service provider, or a software-as-a-service", and state that "Certain features of the Software may be offered for a fee". LM Studio also offers a headless daemon called llmster for servers. Open WebUI's licence, effective with v0.6.6 of 19 April 2025, forbids altering or removing its branding except for deployments with "50 or fewer users in a 30-day period" or with permission, and allows deployment "to any number of internal users within your own organization (with all official branding kept)". The model files carry their own licence, which our comparison of open LLM licences for company use sets out. Whether a clause applies to your company is for your legal counsel to confirm.

What carries over to the company server, and what does not

Our summary; vLLM GGUF documentation (latest, 12 June 2026); Open WebUI README; Ollama library page for gpt-oss; read 25 October 2026.

The model is the most valuable result of the experiment. Ollama's gpt-oss:120b is a 65 GB download, the same open model that OpenAI publishes as about 65 GB of files on Hugging Face, in the MXFP4 format that Ollama runs "natively without additional quantizations or conversions". On the server the engine loads the maker's files. Most other models in Ollama's library come as GGUF files, a format of the llama.cpp project, and vLLM's documentation states that "GGUF support in vLLM is highly experimental and under-optimized at the moment", so the server uses the original or an FP8 or NVFP4 version of the model instead.

Open WebUI connects to "any OpenAI-compatible API alongside local Ollama models", so the chat page your colleagues know can stay and point at the new engine. Its pilot database is a different matter: Open WebUI does not migrate data between databases, as our guide to Open WebUI scaling notes, so decide whether pilot chats move or the company service starts empty.

The hardware step from a PC to a server

The PC experiment usually ran a small model, or a large one slowly. The company server is sized by the model your test preferred and by the number of conversations in progress at the busiest moment. Our users-per-card rule takes 90 per cent of the 95.6 GiB the driver reports on a 96 GB card, minus 3 GiB per card, minus the weights.

All cards are NVIDIA RTX PRO 6000 Blackwell (Max-Q or Server Edition). Conversations of 32,768 tokens in progress at the same moment by our users-per-card rule, from our company-size guide; power by our estimate (servers: cards at 600 W, two processors, plus 15 per cent; workstation: two 300 W Max-Q cards, one processor, plus 15 per cent, as in our two-card Max-Q workstation guide).

For a company of 20 to 100 people, the busy-hour peak is a handful of conversations at once with the example values of our company-size guide, so one card running gpt-oss-120b carries it. The entry server with two RTX PRO 6000 Server Edition cards keeps the second card for the document search models, split by MIG (NVIDIA's Multi-Instance GPU) into up to four separate 24 GB parts. A pair of cards sharing one copy of the model holds about 93 conversations, and a four-card server running two such pairs about 186, which leaves room for a larger model or a second department.

The workstation with two RTX PRO 6000 Max-Q cards runs on an ordinary 230 V socket and suits a team that is not ready for a server room. As of October 2026 we found no maker's noise declaration for such a tower under load, so it belongs in a side room with a door rather than next to desks. NVIDIA allows up to four of these 300 W blower cards in one workstation, and our guide to buying a workstation with two RTX PRO 6000 Max-Q covers it. In a rack server that runs continuously we recommend the Server Edition, because many server makers do not list the Max-Q for their GPU servers. A 230 V 16 A feed carries 2.94 kW at 80 per cent, enough for the two-card server; the four-card server needs 32 A or three-phase.

Ollama lists the RTX PRO 4000, 4500, 5000 and 6000 Blackwell among its supported cards, so the pilot can keep running on the new server while the IT team brings up the server engine next to it.

The engine, versions and network for the IT team

1. Engine. Serve the production model with vLLM (0.32.0 of 18 October 2026, or later) or SGLang; both batch requests continuously and offer an OpenAI-compatible API. Keep Ollama for trying new models, on a separate port and at 0.35.0 or later.
2. Network. Keep every engine on an internal network behind the chat front end or an LLM gateway with keys and a query log. vLLM's --api-key protects only paths under /v1, /v2, /inference and /cohere, and routes such as /metrics answer without it. Our guide to securing a GPU server lists the other layers.
3. Ollama in containers. The official image sets OLLAMA_HOST=0.0.0.0:11434; do not publish port 11434 to the network. Set OLLAMA_NUM_PARALLEL and OLLAMA_CONTEXT_LENGTH explicitly, because the FAQ gives a default context of 4,096 tokens while the context-length page sets 256K for cards with 48 GiB or more.
4. Open WebUI. Run the current release (v0.12.4 on 25 October 2026; v0.12.2 fixed several access issues), set DEFAULT_USER_ROLE=pending or connect single sign-on, and move from SQLite to PostgreSQL before several instances run.
5. GPU selection. Ollama loads a model on one GPU if it fits there and otherwise spreads it across all GPUs; limit it with CUDA_VISIBLE_DEVICES, using GPU UUIDs from nvidia-smi -L, so it does not take the cards of the production engine.
6. Software on delivery. The server arrives on request with the operating system, the NVIDIA driver, CUDA and a container runtime.

We build the two- and four-card servers and set up engine, front end and logins through our Private AI/ML service; send us the model your test preferred and the number of users, and we size the server for it.

What we supply

You can buy this system from us: our team sizes the configuration to your workload and budget and replies within one business day. We build workstations with two RTX PRO 6000 Max-Q cards and servers with two, four or eight RTX PRO 6000 Server Edition cards, assembled and tested before delivery, delivered anywhere in the EU on one EU contract and invoice, with manufacturer warranty on every component handled through us; quantities and lead time are confirmed in writing with the quote. Our Private AI/ML service, with our engineering partner Vixen.UNO, moves the model and the chat front end from the experiment to the server, connects the logins and trains your team to run it. Firms without IT staff will find the division of tasks in our guide to an AI server without an IT department.

FAQ