Preparing company documents for an AI assistant: files, versions, access, exclusions

What an AI assistant needs from your documents

An AI assistant that answers questions from your own company documents can only use the files it is given, in the state they are in. Before it goes live, the people who own the documents choose the folders, remove old versions and duplicates, name an owner for each collection, decide which staff may see which collection, and keep personnel files and other sensitive records out.

Scans need text recognition, Excel files work for looking up values but not for totals, and every document should carry a date that the assistant can show. This guide is for the office manager, the head of quality or the head of R&D who owns those documents. It explains the preparation in plain terms and ends with what the IT side does. The search technology itself, how documents are split and how the search is tuned, is covered in our article on RAG on company data.

How chat with company documents works

The method is called retrieval-augmented generation (RAG). It works in four steps:

1. Conversion. A converter turns every file into plain text with its headings and tables. Scanned pages go through optical character recognition (OCR), which reads the letters from the image.
2. Indexing. The text is cut into passages of a few paragraphs, and each passage is stored in a search index with its document name, date and owner.
3. Search. When someone asks a question, the assistant finds the passages that fit best.
4. Answer. The language model writes the answer from those passages and names the document it used.

Two consequences matter for preparation. A document that was never converted cleanly cannot be found, however good the model is. And the assistant answers from whatever it finds: if a superseded price list is still in the folder, it will one day quote it. The language model reads the passages at answer time and keeps nothing of them afterwards, which is also why a change to a document shows up in answers once it is indexed again. Training a model on company data is a different project, compared in our article on fine-tuning or RAG.

Which files the assistant can read

Docling is an open-source document converter, started at IBM Research Zurich and now hosted by the LF AI & Data Foundation, under the MIT licence. It runs entirely on your own server, and its README names "local execution capabilities for sensitive data and air-gapped environments". Open WebUI, a widely used chat front end, lists Docling and Apache Tika among its document extraction engines. Docling's list of formats shows what such a converter reads.

Docling "Supported formats" page and README on GitHub, read 25 October 2026; Open WebUI README, read 25 October 2026.

Scans. OCR works on clean, upright pages with good contrast. Handwriting, stamps and faint copies come out with errors, so check a sample of your oldest scans before adding a whole archive. Where scans matter most, a newer method searches the page images directly, as our guide to visual document retrieval describes.

Excel. A converter reads a sheet as a table, and the assistant can find the row that answers "what is the lead material of part 4711". It is weak at "what was the total of all orders from customer X in March", because the answer needs a calculation over many rows. For such questions the data belongs in a database that the assistant queries, as our guide to text-to-SQL on-premise explains. Before indexing, give each sheet one table with a header row, and remove hidden sheets and old tabs.

Drawings. CAD formats are not on Docling's list. Where staff need the information in a title block or the notes on a drawing, export a PDF and test with a few examples whether the text comes through.

Duplicates, old versions and owners

This part needs the most time from the people who own the documents, and no software does it for them.

• One current version. Keep only the valid version of a procedure, a price list or a datasheet in the assistant's folders. Move superseded versions to an archive that is not indexed. Our RAG guide puts it plainly: if the old version is still in the index, it will eventually be retrieved.
• No copies in several places. The same document in three folders appears three times in the search results and pushes other passages out. Pick one location.
• A date and a version in the document. The assistant shows the source with each answer, and a visible date lets the reader notice an old answer at once.
• An owner per collection. Name one person per folder who adds new documents, removes old ones and answers questions about the content. Without an owner a collection ages within months.
• Clear file names. "Work instruction welding WI-012 rev C 2026-09" is found and understood; "scan_0045.pdf" is not.

Open WebUI's knowledge feature can mirror a folder: its "Sync directory" action touches only new, modified and deleted files and keeps the folder structure, and for thousands of files its documentation points to a separate sync tool. That keeps the index in step with the folder, which makes a tidy folder all the more important.

We set up the assistant, the converter and the search on your server; send us a list of the folders you have in mind, with rough page counts, and our team proposes the configuration and the order of collections.

Who may see what

The question with the most weight is which staff may see which documents. A file share has permissions per folder; whether the assistant takes them over depends on how it is built.

In Open WebUI, access is set per knowledge base, its name for a document collection: a collection is private or restricted and is shared with user groups or individuals at read or write level. Rights from several memberships add up, and there are no deny rules. The folder permissions of your file server are therefore not copied in automatically. Larger set-ups can store the permissions of each source document in the index and filter every search by the asking user, as our RAG guide describes; this is how our Private AI/ML service builds assistants that respect each user's access rights.

For the document owners the preparation is the same either way: sort documents into collections by audience before anything is indexed.

Our example; access levels as in the Open WebUI documentation on knowledge bases and access, read 25 October 2026.

Customer drawings and documents under a non-disclosure agreement belong in project collections that only the project team can open; our guide to trade secrets, NDAs and AI tools covers the contractual side.

What to keep out of a general assistant

Some documents do not belong in an assistant that many staff can ask, even when they are well organised. This is general information, not legal advice; your data-protection officer confirms what applies to you.

The European Commission's guidance lists as sensitive personal data under the GDPR: data revealing racial or ethnic origin, political opinions, religious or philosophical beliefs, trade-union membership, genetic data, biometric data processed to identify a person, health-related data and data concerning a person's sex life or sexual orientation. The same guidance asks that personal data be "adequate, relevant, and limited to what is necessary for the purpose", that "Data must be stored for the shortest time possible", and that data held is "accurate and kept up-to-date".

In practice that keeps these out of a general assistant:

• personnel files, salary lists, appraisals and applications;
• sick notes, accident reports with health details and occupational-health records;
• works-council and union correspondence;
• email archives that have not been sorted, because they mix personal messages, customer data and attachments;
• customer and supplier data that the assistant's purpose does not need.

Where an HR team wants an assistant for its own documents, that is a separate collection with its own access, its own purpose and usually a data protection impact assessment, as our DPIA guide describes. Where documents mix useful content with personal data, names can be removed before indexing, as our guide to PII redaction before an LLM shows. Our overview of local AI and the GDPR explains what running the assistant on your own server changes and what it does not.

Start with one pilot folder

1. Pick one collection that many people ask about and one owner for it, for example the quality manual and work instructions of one plant.
2. Remove old versions and duplicates, check that each document shows its date, and convert a sample of scans to see how they come out.
3. Collect 50 to 100 real questions with the right answer and the document it comes from, from the people who use the collection.
4. Let the IT side index the folder and run the questions; the owner grades the answers and notes every answer with a wrong or missing source.
5. Fix the documents behind the failures first, then the search settings, and only then add the next collection.

The question set is the most useful result of the pilot, because the same questions show later whether an update made answers better or worse. Our guide to what a private AI pilot should measure gives the metrics. Our Private AI/ML service runs such a pilot on one process with clear metrics; describe the collection you would start with, and we propose the pilot and the server for it.

What the IT side does, and the server it needs

The IT team or the service provider installs the converter, the search index and the models, connects the logins, schedules re-indexing and keeps the query log. This section sums up the hardware.

Reference configurations on our AI servers page; OCR rate about 0.69 pages per second per card with a 5B OCR model, by the estimate in our document AI guide; MIG up to four 24 GB instances per NVIDIA; power by our estimate (cards at 600 W, two processors, plus 15 per cent).

By the users-per-card rule of our company-size guide, one RTX PRO 6000 Server Edition holds gpt-oss-120b with about 19 conversations of 32K tokens in progress at once, which covers the busy-hour peak of a few hundred employees with that guide's example values. The search models are small: the 0.6B embedding and reranking models in our company-size guide fit a 24 GB MIG part. A one-off archive of scans is the heaviest job: 50,000 pages take about 20 hours on one card at 0.69 pages a second, by our estimate, and run overnight or over a weekend. The index needs fast storage; at four bytes per value, one million passages with 1,024-number vectors take about 4.1 GB before index structures and the passage text, as our company-size guide calculates.

The models and documents stay on the server in your network, or on dedicated hardware in a Tier-3 data centre in Lithuania through our Private AI/ML service, inside the EU. Both servers belong in a server room for noise and heat. The RTX PRO 6000 Max-Q is the 300 W workstation card; for a rack server that runs continuously we recommend the Server Edition.

What we supply

You can buy this system from us: our team sizes the configuration to your workload and budget and replies within one business day. We build the servers to order with two, four or eight RTX PRO 6000 Server Edition cards, assembled and tested before delivery, delivered anywhere in the EU on one EU contract and invoice, with manufacturer warranty on every component handled through us; quantities and lead time are confirmed in writing with the quote. Our Private AI/ML service, with our engineering partner Vixen.UNO, builds the assistant on your documents with citations and access rights, a query log, and the pilot on your first collection.

FAQ