AI server for audit and accounting firms: receipts, ledgers and audit files in-house

What an AI server does for an audit or accounting firm

An AI server for an audit or accounting firm runs document, language and search models on hardware the firm controls, so client records, ledgers and working papers stay under the firm's own access rules. It reads receipts, invoices and contracts into structured records, drafts explanations for ledger entries that the firm's tests flag, summarises the documents of an engagement for the audit file and answers staff questions from the firm's methodology. For a mid-size firm, a server with two NVIDIA RTX PRO 6000 Blackwell Server Edition cards carries the assistant and the document pipeline; four cards carry the busy season after year-end.

With four cards, one card can take the scanned documents and the others keep room for long reviews. The figures below are published measurements with their source, or our estimates, labelled as such. We wrote this for the managing partner who decides whether such a server is worth it, and for the IT team that runs it next to the audit software and the document archive. The legal notes are general information on EU law as of October 2026, not legal advice; national professional rules decide the details, and your professional body, data-protection officer or legal counsel confirms them.

Four workloads and what sizes each

Our summary; pages by our estimate of 0.69 pages per second per card (a full day of continuous operation); model sizes and cache by our users-per-card rule.

Documents. Receipts, supplier invoices, bank statements, contracts, confirmations and minutes arrive as scans and PDFs. As our guide to vision-language models for document AI explains, a two-stage pipeline works well: an OCR model turns each page into text with its tables, and a text model fills a fixed JSON schema with date, supplier, amounts, VAT and account, which vLLM's structured outputs enforce. The bookkeeping system or the audit tool imports the record and checks it.

Ledger review. Journal-entry tests such as postings at weekends, round amounts, unusual users or entries after closing are rules over the whole ledger; they run in the database or the audit tool and give the same answer every time. The language model gets the entries the tests flag, with account names and booking texts, and drafts a note for the working paper that a person checks. It does not compute balances: totals come from code, and the model only explains them.

Audit-file summaries. Each engagement collects documents that someone has to read and summarise: board minutes, loan agreements, lease contracts, legal letters. Summaries are a batch job, which our guide to batch LLM inference on-premise sizes in GPU-hours.

Methodology and long reviews. Staff ask how the firm's methodology treats a case and which template applies; the model answers from the firm's manuals and cites them, as in RAG on company data. A reviewer who loads a pack of contracts and minutes into one context needs far more GPU memory per request than a short question.

Professional secrecy, the audit file and the GDPR

For statutory audits, Directive 2006/43/EC sets the EU frame. We read the consolidated text of 9 January 2024 on EUR-Lex on 24 October 2026. Article 23(1) requires Member States to ensure that "all information and documents to which a statutory auditor or audit firm has access when carrying out a statutory audit are protected by adequate rules on confidentiality and professional secrecy". The rules themselves are national, and whether a public AI service may receive such documents is a question for those rules and the firm's engagement terms. A model on the firm's own server keeps documents, prompts and answers on hardware under the firm's access rules, with no external service unless the firm switches one on.

Article 24b(5) asks the auditor to "create an audit file for each statutory audit", to retain "any other data and documents that are of importance in support of the report", and closes the file "no later than 60 days after the date of signature of the audit report". For the platform this means two things. Where a model's summary or note supports a conclusion, the firm's methodology decides whether it goes into the file, and the platform must let staff export it, with the source documents it cites, before the file closes. After that, the copies on the AI platform should not become a second, unofficial audit file: set a retention for the platform's index and logs per engagement and delete them on schedule.

Accountancy Europe, in its comments of 18 November 2024 on a US paper about generative AI in auditing, names the risks in one line: "Key concerns include data reliability, algorithmic biases and confidentiality issues, and require particular attention." The first concern is why the ledger arithmetic stays in code, and the last is why access follows the engagement. In the retrieval index every chunk carries the client, the engagement and the access list of its source, and retrieval filters before it ranks, so an engagement team never sees another client's documents. Our guide to a private LLM for law firms and tax advisers describes the same design per matter for the advisory side of a practice.

The GDPR applies to personal data in client records, payroll documents and logs wherever the model runs, and Article 28 governs any provider that processes them on the firm's behalf. Under the AI Act, a firm that uses an AI system is a deployer; our article on the EU AI Act for companies using LLMs covers the duties that apply to an internal assistant.

We supply the server and set up the document pipeline and the search per engagement through our Private AI/ML service; send us your number of engagements and pages per season, and our team replies with a configuration.

E-invoices change the document mix

Council Directive (EU) 2025/516 of 11 March 2025 amends the VAT Directive "as regards VAT rules for the digital age". We read it on EUR-Lex on 24 October 2026. The amended Article 217 defines an electronic invoice as one that contains the required information and has been "issued, transmitted and received in a structured electronic format", and the amended Article 218(2) states that "invoices shall be issued as electronic invoices". Member States apply that part, Article 5 of the amending directive, from 1 July 2030, and Article 1 already allows them to require electronic invoices for domestic supplies. Member States that had a domestic digital real-time transaction-based reporting obligation in place on 1 January 2024, or had obtained the Council's permission or adopted national legislation to introduce one before that date, apply the Article 218 change to domestic invoicing and reporting by 1 January 2035 (Article 6(5)).

A structured invoice is data, and a parser reads it without a GPU. As the share of structured invoices grows, the OCR load of a bookkeeping firm moves to receipts, documents from suppliers outside the scheme, contracts, bank and legal confirmations and the archives of earlier years. Size the OCR card for those and for the years of paper your clients will still bring. How the rules apply to a particular client's invoices is a question for a tax adviser; our guide to buying IT hardware from another EU country covers VAT for the hardware purchase itself.

Sizing the busy season

The season after year-end close sets the size. Take, as example values, a firm with 120 audit engagements and about 1,500 pages of scanned supporting documents each: 180,000 pages. Datalab reports 1.44 pages a second for its 5B chandra-ocr-2 on one H100 80GB with vLLM; scaled to the RTX PRO 6000 Server Edition by memory bandwidth, that is about 0.69 pages a second per card by the method of our document AI guide. One card then needs about 73 hours, six nights of 12 hours; four cards running one copy each need about 18 hours.

OCR at about 0.69 pages per second per RTX PRO 6000 Server Edition, our estimate scaled from Datalab's chandra-ocr-2 figure on one H100 80GB (model card, undated) by the method of our document AI guide; summaries from MLPerf® Inference: Datacenter v6.1, closed division, available category, gpt-oss-120b, offline scenario, entry 6.1-0050 (HPE ProLiant Compute DL380a Gen12, eight RTX PRO 6000 Blackwell Server Edition), 15,256 tokens per second, retrieved from mlcommons.org on 24 October 2026, result verified by MLCommons Association; 1,907 per card is our division of the system result, not an MLPerf metric, and we plan with half of it; volumes are example values; long reviews by our users-per-card rule. The MLPerf name and logo are registered and unregistered trademarks of MLCommons Association in the United States and other countries. All rights reserved. Unauthorized use strictly prohibited. See www.mlcommons.org for more information.

A bookkeeping practice has a monthly rhythm instead. With 400 clients, about 150 receipts per client a month and 1.2 pages per receipt, it receives 72,000 pages a month, about 3,400 per working day over 21 days, which one card converts in about 82 minutes by our estimate.

Summaries follow the method of the batch guide. 120 engagements with 300 documents each give 36,000 summaries; at about 500 output tokens each that is 18 million tokens. At 1,907 tokens per second per card, our division of the MLPerf result, one card needs about 2.6 hours; with half the published rate as our planning margin, about 5.2 hours, one night. Contracts are longer than the benchmark's prompts, so reading them adds time; time a sample of your own documents first. The licence comes before the speed: GLM-OCR is published under MIT and DeepSeek-OCR-2 under Apache 2.0, while chandra-ocr-2 carries a modified OpenRAIL-M licence with a revenue threshold.

Long reviews decide the memory. By our users-per-card rule, which takes 90 per cent of the 95.6 GiB the driver reports, less 3 GiB per card and less the weights, gpt-oss-120b leaves 22.2 GiB of cache on one card. A 32K request takes 1.125 GiB of 16-bit cache and a 128K review 4.5 GiB, so one card holds 19 questions or 4 reviews, and a tensor-parallel pair, with 105.3 GiB of cache, 93 questions or 23 reviews. Staff of a firm in its busy season ask more than an average employee; with 8 requests in flight per 100 staff, the example value our consulting guide uses, 150 staff need 12 at the peak, within one card.

Configuration options by card count

The options follow the reference configurations on our AI servers page and NVIDIA's guide for certified systems: two processors, at least six cores per GPU and system memory of twice the GPU memory.

Reference configurations on our AI servers page; requests and reviews by our users-per-card rule with gpt-oss-120b and a 16-bit cache; pages by our estimate of 0.69 per second per card over a full day; power by our estimate with the cards at 600 W, processors and 15 per cent for the rest of the server.

Two cards. The entry server has two RTX PRO 6000 Server Edition, two AMD EPYC processors, 384 GB of DDR5 ECC and two NVMe drives. Card 1 runs gpt-oss-120b for questions and reviews by day and the summary batch at night. Card 2 runs the OCR model and the embedding model and reranker side by side; the retrieval models are small, so both fit the 96 GB card. Where the firm wants the retrieval models isolated, MIG splits card 2 into four 24 GB instances, at a lower OCR rate for which no figure is published. This size suits a firm with a few dozen engagements or a bookkeeping practice.

Four cards. The inference node with 768 GB and four NVMe drives runs the assistant as one copy split over cards 1 and 2 with tensor parallelism, card 3 converts documents through the season and card 4 holds a second copy of the model for summaries and drafting by day, so the batch no longer waits for the night. We would size this first for a firm whose season is the bottleneck.

Two four-card servers or eight cards. Where staff depend on the platform in the weeks before the reporting deadlines, two servers that each carry the peak keep it running while one is serviced, as our guide to LLM high availability with two GPU nodes describes. Eight cards in one server give the same capacity in one chassis, at about 6.3 kW by our estimate and without failover.

The RTX PRO 6000 Max-Q is the workstation card, a 300 W blower card; in a rack it belongs only in a chassis whose maker lists it, and for continuous operation we recommend the Server Edition. We set up the models through our Private AI/ML service; describe which workload you would start with, and we propose a pilot on one engagement type and the server for it.

Notes for the IT team

1. Numbers stay in code. Ledger tests, totals and reconciliations run in the database or the audit tool. The model receives flagged rows and returns text; a check compares every figure in its note with the source row.
2. Schema for every extraction. vLLM's structured outputs make the answer follow a JSON schema, so the import rejects a record with a missing field instead of guessing. Keep pages that the OCR stage reads with low confidence for a person.
3. Access per engagement. Directory roles map to engagements; the retrieval filter checks the access list before ranking. Put an LLM gateway with keys and a query log in front of the engines and log user, engagement, sources and output.
4. Export before closing. Give staff an export of the notes and summaries they rely on, with the cited documents, into the audit file, and a deletion job for the platform's copies after the file closes.
5. Untrusted documents. Client documents can carry text aimed at the model. Extraction into a schema and a model without tools limit the damage, as our guide to prompt injection and LLM security explains.
6. Network and versions. vLLM's --api-key protects only paths under /v1, /v2, /inference and /cohere, while routes such as /invocations and /metrics answer without the key (vLLM 0.32.0 security guide), so keep the engines on an internal network behind the gateway. Run vLLM 0.31.0 or later, the NVIDIA Container Toolkit at 1.17.8 or later and a driver NVIDIA lists as fixed in its bulletin of 30 September 2026.

Power and room

By our estimate, the two-card server draws about 1.8 kW at the wall with two EPYC 9135 processors and fits one 230 V 16 A feed, which carries 2.94 kW at 80 per cent. The four-card server draws about 3.4 kW with the cards at 600 W and needs 32 A or three-phase, or about 2.7 kW with the cards capped at 450 W. Eight cards draw about 6.3 kW by our estimate; Dell's datasheet gives about 8 kW per eight-card node as an indicative figure, and feeds should be planned on the maker's figure. A measurement on the delivered system confirms the draw. Our guide to GPU server power on 16 A, 32 A and three-phase feeds works through the feeds. Without a server room of your own, our Private AI/ML service runs the platform on dedicated hardware in a Tier-3 data centre in Lithuania, inside the EU.

What we supply

You can buy this system from us: our team sizes the configuration to your workload and budget and replies within one business day. We build the servers to order with two, four or eight RTX PRO 6000 Server Edition cards, assembled and tested before delivery, delivered anywhere in the EU on one EU contract and invoice, with manufacturer warranty on every component handled through us; quantities and lead time are confirmed in writing with the quote. Our Private AI/ML service, built by our engineering partner Vixen.UNO, deploys the document pipeline, the models, the search per engagement and the query log. Our guide to buying a GPU server for LLMs lists every configuration.

FAQ