AI server without an IT department: who does what, what the room needs, what if it fails

Running an AI server without an IT department

A company without an IT department can run its own AI server if three roles are filled: one person in the firm who owns the assistant and needs no IT training, the external IT service provider that already looks after the network and PCs, and a support provider that keeps the server, its software and its models up to date under an agreed SLA. The building needs a suitable place with power, cooling and a network port. When a part fails, the support provider diagnoses it remotely and the warranty case goes through the supplier.

The concern is common. Eurostat's survey of 2025 found that among EU enterprises that had considered using AI, 70.89 per cent named a lack of relevant expertise as a reason for not using it, the most frequent reason in the survey, ahead of unclear legal consequences at 52.52 per cent and data protection concerns at 48.83 per cent. This article is for the owner or managing director of such a firm. It explains, in plain terms, what each role does from day to day, what happens when something breaks and what to keep ready. For the size of the machine, see our guide to an own AI server for a company of 10 to 100 people; for larger firms with an IT team, the operating models with a full responsibility matrix are in our guide to managed private AI.

What the building needs

A small firm's AI server is a single machine. It still needs a place that suits it, and the checks are simple enough to do with your electrician and your IT service provider.

Our guides to a GPU server in an office and to preparing a server room, with ASHRAE's 2021 Equipment Thermal Guidelines as cited there; power by our estimate.

By our estimate, a workstation with two NVIDIA RTX PRO 6000 Blackwell Max-Q cards draws about 1.1 kW at the wall and a rack server with two NVIDIA RTX PRO 6000 Blackwell Server Edition cards about 1.8 kW. Both fit one 16 A circuit if nothing large shares the breaker. The heat they produce equals that power, so a room that is comfortable for one person can become too warm with the machine running. Our guides to a GPU server in an office and to preparing a server room go through each point. If there is no suitable room, the models can run on dedicated hardware in a Tier-3 data centre in Lithuania through our Private AI/ML service, and the building question disappears.

Three roles and their tasks

The table splits the recurring tasks between the three roles. It is an example to adapt: some IT service providers take on the support role themselves, and some firms keep more in-house.

Our example split for a firm without IT staff; the operating tasks follow our guides to managed private AI and to operating a private LLM platform.

The contact person is often the office manager, the head of quality or the person who pushed for the project. They decide which documents the assistant may use and for whom, request accounts for new staff and close them for leavers, collect the questions the assistant answered badly and pass them on. They do not need to know Linux.

The IT service provider that already looks after your network, firewall, PCs and user accounts keeps doing that. For the AI server it adds a network address, a separate connection for the remote management, the accounts in your directory service if you have one, and the server in its backup plan.

The support provider looks after everything on the machine itself: the operating system, the NVIDIA driver, the serving engine that runs the models, the models, the web interface and the document search. With our Private AI/ML service this is our team: we build and support the platform, train your people to work with it, and afterwards the choice is yours, your own staff or our support under an agreed SLA. Tell us who looks after your IT today, and we propose a division of tasks that fits it.

A normal week for the contact person

Most weeks the contact person spends little time on the server. A new colleague needs an account; a department wants its procedures added to the document search, and the contact person checks that the files are the current versions and agrees who may see them; someone reports that the assistant quoted an old price list, and the contact person removes the old file from the collection and tells the support provider. Once a month, or when the support provider asks, they agree a maintenance window for updates.

What the contact person should not do is change settings on the machine, install software on it or move it. Those tasks belong to the support provider, so that the machine stays in the state that is documented.

AI server maintenance: updates and remote access

An AI server needs more regular updates than a file server, because several of its parts change often. NVIDIA, according to its driver lifecycle page as quoted in our managed private AI guide, releases two production driver branches a year, each with bug fixes and security updates for up to one year, beside long-term branches such as R580, supported until June 2028. vLLM, a widely used serving engine, states in its release document that it aims at a regular release every two weeks, and it published more than 20 security advisories between late August and early October 2026, as the same guide records.

None of this has to be applied the day it appears. The support provider reads the notes, decides with you what is needed, tests it and applies it in the agreed window. A model change is your decision, because it changes the answers staff get; the support provider runs your set of test questions on the new model first, as our guide to LLM evaluation before model upgrades describes.

The support provider reaches the machine through two paths. The first is the normal network connection to the operating system, over a connection you control. The second is the BMC (baseboard management controller), a small separate computer inside the server with its own network port. It shows temperatures, fan speeds, power supplies and fault messages, and it can restart the machine, even when the operating system has stopped. Our guide to GPU out-of-band monitoring explains what it reads. Rack servers have one; for a workstation, ask for a platform that offers it. The management port belongs on a separate internal network and never on the internet, as our guide to securing a GPU server explains.

When something fails

Most faults show up as an assistant that is slow, gives errors or does not answer. The contact person reports what happened and when; the support provider looks at the machine remotely and decides what to do.

Our summary of typical fault cases; the model on the remaining card by our users-per-card rule; redundant power supplies as listed for our GPU server chassis.

One GPU. On a machine with two RTX PRO 6000 cards, the chat model gpt-oss-120b fits on one card, so by our estimate the support provider can move it to the remaining card until the replacement arrives, with the document-search and speech models paused. The faulty card is identified by its slot and serial number, as our guide to identifying a faulty GPU shows for IT staff.

Power supply and drives. Our GPU server chassis have redundant power supplies, so a rack server keeps running on the second one while the failed one is replaced. A workstation often has a single supply, and then it stops until the supply is changed; ask at the order whether a second one is possible. Whether the server keeps running after a drive failure depends on a choice made before delivery: with the two NVMe drives set up as a mirror, one can fail without data loss. Ask for it in the quote.

Heat. GPUs reduce their speed when they get too hot; the NVIDIA driver reports this as throttling, and the BMC shows the temperatures. In an office, dust on the air intake, a blocked grille or a room cooler that switches off at night can be the cause. Because the BMC shows the temperatures, the support provider sees the trend before staff notice.

We supply the machine with manufacturer warranty on every component, handled through us, and check the room, power and airflow before we quote; send us the room and the circuit you have, and we reply with a configuration within one business day.

The warranty path and what to keep ready

When a part is faulty, you do not need to find out which maker is responsible for it. The manufacturer warranty on every component is handled through us, units that are defective on arrival are replaced, and the assembly itself is covered by us. The support provider runs the diagnostics the manufacturer asks for and opens the case with us.

Keep these in one folder, on paper and in a shared drive that does not depend on the AI server: the delivery note and invoice, the serial numbers of the machine, each GPU and each drive, the contact addresses for faults, the access data for the remote management in a safe place, and a short record of each fault with date, time and what was observed.

The drives need a decision before the first fault. They hold the document index, the query log and possibly copies of your documents. Ask in the quote what happens to a failed drive: whether it has to go back to the maker or may stay with you, and if it goes back, how it is erased first. Our guide to the GPU server lifecycle covers erasure under NIST SP 800-88 Rev. 2 and the certificate per drive; your data-protection officer decides what your firm requires.

Backups: what needs one

Not everything on an AI server needs the same backup. The configuration of the platform, meaning the settings, the access mapping and the deployment files, is the part that is hard to rebuild; it belongs in a repository or share that your firm owns. The query log is kept for as long as your rules say and backed up with it. The documents themselves usually stay in their original place, on the file server or in the document system, and the search index can be rebuilt from them. Model files can be downloaded again from their publisher, but keep a copy of the exact version in use, so that a restore gives the same answers. Your IT service provider adds these items to the backup it already runs; our article on the 3-2-1-1-0 backup rule explains how many copies to keep and where.

Outside access and data protection

A provider with administrator rights on the AI server can read what staff type, the answers, the documents in the index and the query log, which often contain personal data. Article 28(3) of the GDPR states that "Processing by a processor shall be governed by a contract or other legal act under Union or Member State law", which sets out the subject matter, duration, nature and purpose of the processing, the types of personal data and the rights and obligations of the controller. Whether your IT service provider or your support provider acts as a processor for your data is an assessment for your data-protection officer or legal counsel. This is general information, not legal advice.

Three practical rules help either way: give each outside provider named accounts, not shared ones; log every session on the machine; and revoke the access on the last day of a contract. When our engineering partner Vixen.UNO works on your platform, its engineers work in your environment under your access controls and take no copies of production data out of it, with an NDA before technical detail and a data processing agreement on request. Our guide to local AI and the GDPR covers what own hosting changes and what stays your duty.

What we supply

You can buy this system from us: our team sizes the configuration to your workload and budget and replies within one business day. We build AI workstations and servers to order, typically with two RTX PRO 6000 Max-Q in a workstation or two RTX PRO 6000 Server Edition in a rack server for a small firm, assembled and tested before delivery, delivered anywhere in the EU on one EU contract and invoice, with manufacturer warranty on every component handled through us; quantities and lead time are confirmed in writing with the quote. On request we install the operating system, drivers, CUDA and a container runtime and commission the machine on site. The models, documents, training and support under an agreed SLA are our Private AI/ML service, with one Eurokommerz project lead as your contact. The steps from the first conversation to daily use are in our guide to the AI server project from request to use.

FAQ