Private AI RFP: the questions to put to every vendor before you buy an LLM platform
Eurokommerz, Vienna, since 2006: Private AI/ML · IT Managed Services · Enterprise Training · AI Hardware & Software
- A private AI RFP covers the whole project and puts the same questions to every vendor in eight areas: data, models, answer quality, operations, security and compliance documents, ownership, exit and cost structure; the GPU server gets a technical specification of its own
- Quality is judged on the buyer's own documents: a fixed set of questions with accepted answers and pass marks written down before the pilot, so that every offer faces the same test
- Under the AI Act as amended by Regulation (EU) 2026/1744, in force since 27 July 2026, Article 4 on AI literacy has applied since 2 February 2025 and Article 50 since 2 August 2026, while the high-risk rules for Annex III uses apply from 2 December 2027; this is general information and no legal advice
- ISO/IEC 42001:2023, the NIST AI Risk Management Framework 1.0 and the OWASP Top 10 for LLM Applications 2025 give an RFP a shared vocabulary; ask each vendor which of them it works to and what evidence it can show
- You can order the hardware alone, the setup of the software alone, or both together: the server delivered with the models and the assistant installed and configured
Eurokommerz × Vixen.UNO: Private AI/ML Talk to an expert →
What a private AI RFP has to ask
A private AI RFP, the request for proposal for an LLM platform that runs on-premise or on dedicated hardware, puts the same questions to every vendor in eight areas: data, models, answer quality, operations, security and compliance documents, ownership, exit and cost structure. Each question should ask for a document, a figure or a demonstration, so that the answers can be laid side by side. The two tables below can be copied as an AI RFP template and adapted to your case.
These questions concern the whole project. The GPU server gets a technical specification of its own, with measurable minimums for memory, bandwidth, power and warranty, and our guide to the GPU server tender specification covers it. A buyer who first wants to know the size of the hardware finds the ten facts for that in our private AI sizing request.
LLM RFP questions on data, models and answer quality
| AREA | QUESTION TO VENDORS | WHY IT MATTERS |
|---|---|---|
| Data | Where are prompts, documents, the index and the logs processed and stored in each deployment option you offer? | shows whether anything leaves your network |
| Data | Who on your side gets access during setup and support, from where, and under whose access controls? | access to personal data has to be covered by contract before the work starts |
| Data | What is logged, who can read the log and how long is it kept? | the log is the audit trail and at the same time a store of personal data |
| Models | Which models do you propose, in which precision and under which licence? | licence terms differ by model and decide what a company may do with it |
| Models | Who decides on a new model version, who tests it, and how is the previous one kept? | answers can change with every change of model |
| Quality | How are answers evaluated on our documents before acceptance: which questions, which metrics, which pass marks? | makes offers comparable and acceptance checkable |
| Quality | How does retrieval respect the access rights of each user? | no user may receive passages from documents outside their rights |
our own list of questions; the sections below explain each area.
On data, ask for a drawing as well as a sentence. A platform keeps data in at least three places: the prompts and answers in transit, the documents with their index, and the logs. For each place the vendor should name the system, the location and the people with access. Access by the vendor's staff deserves its own answer: where a vendor processes personal data on your behalf, the GDPR requires a contract for it (see the section on the GDPR below).
On models, the licence belongs in the offer. Open models come under permissive licences, custom licences with use policies and licences for non-commercial purposes only, and our comparison of open LLM licences for company use sets them side by side. Ask who keeps a local copy of the weights you serve: base weights can be downloaded again only while that revision exists, which our guide to backing up an AI server explains.
On quality, do not accept a demonstration on the vendor's own data as proof. Build a fixed set of questions from your documents, write the accepted answers and the pass marks down before the pilot, and let every vendor answer the same set. The same set later serves as the regression test for each model or prompt change, as described in LLM evaluation before model upgrades. Where the assistant answers from documents, wrong answers usually start in retrieval, as RAG on company data explains, so the set should include questions whose documents a given user may not see.
We supply the servers for such projects and set up the models and assistants on them; send us your list of questions and we answer it for your case.
RFP questions on operations, security, ownership, exit and costs
| AREA | QUESTION TO VENDORS | WHY IT MATTERS |
|---|---|---|
| Operations | Which metrics are monitored, who receives the alerts and who acts on them? | decides who notices a full queue or a failed card |
| Operations | Who installs updates of the serving engine, the GPU driver and the firmware, and in which maintenance window? | updates continue for the whole life of the platform |
| Operations | How do you define support: service hours, response times by priority, escalation? | each vendor defines support differently, and only stated figures can be compared |
| Security | Against which entries of the OWASP Top 10 for LLM Applications 2025 was the system tested, and how? | prompt injection and disclosure of data are risks specific to LLM applications |
| Security | Which standards and frameworks do you work to, and what evidence can you show? | a named standard can be checked only with evidence |
| Ownership | Who owns the system prompts, evaluation sets, index, adapters and fine-tuned weights created in the project? | they exist nowhere else and carry your knowledge |
| Exit | What stays with us at the end of the contract, in which format, and what stops working without you? | shows the dependence on the vendor before it exists |
| Costs | Which items are one-off, which recur, and with which quantity does each grow? | a total over several years needs the unit of every item |
our own list of questions; the OWASP list is the 2025 edition on the project page, read 12 November 2026.
Support is the area in which offers differ most in wording. Leave the definition to the vendor and ask for figures: the hours in which someone answers, the response time for each priority, the maintenance windows and who approves a change. Then compare the figures; the names of service levels differ from vendor to vendor. Who can run the platform after go-live, your own staff, the vendor or a mix, is the subject of our article on the managed private AI platform.
Security questions with OWASP, ISO/IEC 42001 and NIST as references
An RFP can name three public references, each for a different purpose.
The OWASP Top 10 for LLM Applications is a list of risks. The current list on the project page, read on 12 November 2026, is the 2025 edition: LLM01:2025 Prompt Injection, LLM02:2025 Sensitive Information Disclosure, LLM03:2025 Supply Chain, LLM04:2025 Data and Model Poisoning, LLM05:2025 Improper Output Handling, LLM06:2025 Excessive Agency, LLM07:2025 System Prompt Leakage, LLM08:2025 Vector and Embedding Weaknesses, LLM09:2025 Misinformation and LLM10:2025 Unbounded Consumption. Ask per entry which test was run and with which result. Our article on prompt injection and LLM security describes the controls.
ISO/IEC 42001:2023, "Information technology - Artificial intelligence - Management system", is a management-system standard. According to its abstract, read in a standards catalogue on 12 November 2026, it "specifies the requirements and provides guidance for establishing, implementing, maintaining and continually improving" an AI management system and is "applicable to any organization, regardless of size, type and nature". It describes how an organisation manages AI, so the RFP question is whether the vendor works to it and for which scope.
The NIST AI Risk Management Framework 1.0, released on 26 January 2023, is "intended for voluntary use" and has four functions: Govern, Map, Measure and Manage. NIST states that the framework is being revised. Its Generative AI Profile, NIST AI 600-1 of July 2024, lists twelve risks, among them "Data Privacy", "Information Security" and "Confabulation", which it defines as "the production of confidently stated but erroneous or false content". The profile is a useful source of terms for the quality and security sections of an RFP.
A reference in an RFP is a request for evidence. It does not replace the test on your own documents.
EU AI Act and GDPR points for an AI request for proposal
The following is general information as of 12 November 2026 and no legal advice; your legal department assesses your case. The AI Act is Regulation (EU) 2024/1689 as amended by Regulation (EU) 2026/1744 of 8 July 2026, the Digital Omnibus on AI, in force since 27 July 2026.
| PROVISION | WHAT IT SAYS | QUESTION FOR THE RFP |
|---|---|---|
| AI Act, Article 4 | in the wording amended in 2026, providers and deployers "shall take measures to support the development of AI literacy" of their staff; Article 4 has applied since 2 February 2025 | Which training for users and administrators is part of the offer? |
| AI Act, Article 50 | providers ensure that AI systems intended to interact directly with natural persons inform them; applies since 2 August 2026 | How does the assistant tell users that they are dealing with an AI system? |
| AI Act, Articles 13 and 26 | high-risk systems only: instructions for use from the provider; the deployer assigns human oversight, monitors operation and keeps logs "of at least six months"; for Annex III uses from 2 December 2027 | If a planned use falls under Annex III: which documentation and log functions do you deliver for it? |
| AI Act, Article 53 | providers of general-purpose AI models "draw up, keep up-to-date and make available information and documentation to providers of AI systems" that integrate the model; the obligations for these models have applied since 2 August 2025 | Which documentation of the model provider do you pass on with each proposed model? |
| GDPR, Article 28 | a processor "processes the personal data only on documented instructions from the controller" under a written contract and engages no other processor without "prior specific or general written authorisation" | Will you act as processor, and which subprocessors do you name? |
| GDPR, Article 32 | security appropriate to the risk, including "a process for regularly testing, assessing and evaluating the effectiveness" of the measures | Which technical and organisational measures do you document for the platform? |
| GDPR, Article 35 | a data protection impact assessment "prior to the processing" where it is "likely to result in a high risk" | Which descriptions of the system do you supply for our assessment? |
AI Act articles in the consolidated text of the Commission's AI Act Service Desk, with the dates of Article 113 as amended; GDPR, Regulation (EU) 2016/679; all read 12 November 2026.
Two limits keep these questions in proportion. Articles 13, 16 and 26 of the AI Act concern high-risk systems only, and for the uses listed in Annex III they apply from 2 December 2027; for an internal assistant they are questions about readiness and describe no current duty, and whether a planned use is high-risk is for your legal department to assess. Article 53 binds the provider of the general-purpose AI model; the company that integrates the model is not its addressee. Its paragraph 2 says that points (a) and (b) of paragraph 1 "shall not apply to providers of AI models that are released under a free and open-source licence", except for models with systemic risk, so for an open model ask which documentation exists. Which role your company and the vendor take, and what Article 4 and Article 50 mean in daily use, is explained in our article on EU AI Act deployer obligations; what IT supplies to the assessment under Article 35 of the GDPR is in the DPIA for an internal LLM.
Public buyers can also look at the EU model contractual clauses for AI. The page "Updated EU AI model contractual clauses" of 5 March 2025 in the Public Buyers Community offers "a full version for high-risk AI, aligned with the EU AI Act adopted on 13 June 2024", a light version for non-high-risk AI and a commentary, in English and 23 other EU languages. The clauses are a community resource written for public buyers. They predate Regulation (EU) 2026/1744, so their dates need checking against the amended text.
Ownership and exit at the end of the contract
A project produces things that exist nowhere else: system prompts, the evaluation set, the index built from your documents, adapters and fine-tuned weights, the configuration of the platform and the logs. The RFP should ask who owns each of them and in which format it is handed over. For open models, the licence of the base model can set conditions for what is derived from it, which is one more reason to have the licence named in the offer.
For the exit, ask for a handover list and a rehearsal. The list names the weights and adapters with their SHA-256 checksums, the index or the procedure that rebuilds it, the configuration, the documentation and the administrator accounts. The rehearsal is a restore on your side: checksums of the weight files, the fixed evaluation set and the top results of fixed queries show that the platform runs without the vendor. Where the vendor processed personal data, the contract under Article 28 of the GDPR provides that the processor "deletes or returns all the personal data to the controller"; ask how that is confirmed. For the hardware, the questions on warranty and the supplier are in our GPU supplier checklist and in GPU server warranty in the EU.
Cost structure without figures
Ask every vendor for the same cost table: the item, its unit, the quantity, and whether it is paid once or recurs. Hardware, the technical assessment, implementation and training are usually one-off. Support, subscriptions and power recur. Licences need their unit stated. NVIDIA's licensing guide, last updated on 2 September 2026, says that "NVIDIA AI Enterprise software is licensed on a per-GPU basis", and NVIDIA's NIM FAQ says that "Using NIM in production requires an NVIDIA AI Enterprise license", so the offer should state whether the proposed software stack needs it. Ask also what triggers additional cost: a further model, more users, a new document source, a second site. A comparison with cloud APIs over the same period belongs in the answer, and private LLM vs cloud API shows how to calculate it with your own token volumes.
How we answer these questions
Our answers are those of our Private AI/ML page, and we add nothing to them here.
- Data. Nothing goes to public services unless you explicitly enable it. The models run on your servers or on dedicated hardware in a Tier III data centre in Lithuania; a hybrid mode with public APIs is enabled only by your explicit decision and is visible in the query log. The work is done in your environment under your access controls, and no copies of production data are taken out of it. There is an NDA before technical detail, a data processing agreement on request, and subprocessors are named in the contract.
- Models. Open and commercial models are deployed on-premise with vLLM, Ollama or NVIDIA AI Enterprise.
- Quality. We start with a pilot on one process with clear metrics, measure the result at checkpoints and scale only what has proved its value. Assistants answer on the basis of your documents, cite the source and respect each user's access rights.
- Operations and security. The solution includes a query log, data and permissions management and protection of the models against prompt injection, and your team is trained to run the platform and develop it further.
- Compliance. The legal compliance assessment is done by your legal department; we deliver the technical part.
- Costs and contract. A TCO calculation against cloud GPUs comes before the purchase, the first call is free, the price of the technical assessment is fixed before work begins, and the contract is with us, an Austrian company.
The page states no support hours, no evaluation method in detail, nothing on the ownership of fine-tuned weights and no exit terms. Put these questions to us as to any other vendor. How the hardware grows after the pilot is described in from pilot to production.
You can order the hardware alone, the setup of the software alone, or both together: the server delivered with the models and the assistant installed and configured; write to us and we suggest the best solution for your tasks and budget.
FAQ
What should a private AI RFP include?
Which LLM RFP questions should we ask about our data?
Is there an AI RFP template from the EU?
Does the EU AI Act have to appear in an AI request for proposal?
Who owns the prompts, the index and fine-tuned weights after the project?
How do we compare vendors for on-premise AI?
Does the RFP need a hardware specification for the GPU server?
We supply the hardware for private AI alone, set up the software alone, or deliver both together. Describe your task in a few sentences and our team will help. The first call is free. We reply within one business day.
Talk to an expertWe reply within one business day