Chat with your database: ask ERP, CRM and SQL data in plain language with a local LLM

Asking the ERP in plain language

Chat with your database means typing a business question into a chat window and getting the answer from your own data: a language model translates the question into SQL, the query language of databases (hence "text-to-SQL"), the database runs it, and the answer comes back as a table or chart together with the query. Run locally, the model sits on a workstation or server in your company, so the questions, the structure of your tables and the results never go to a cloud service. For a controlling or sales team, one NVIDIA RTX PRO 5000 Blackwell with 48 or 72 GB is the practical starting point.

This page is written for managing directors, controllers and heads of sales who want answers from the ERP or CRM without waiting for a report, and for the IT colleague who will connect it. Typical questions are "which customers ordered less this quarter than in the same quarter last year", "open orders above a given value by sales rep" or "average days from quote to order by product line". Each needs a few tables joined correctly and a filter that matches how your company counts.

This page covers what a company needs to start: the tools, the business definitions, the access rules and a workstation. The model choice for a BI team with large warehouses, schema retrieval and multi-card servers is covered in our guide to text-to-SQL on-premise. Searching documents is a different task, covered in our guide to RAG on company data: a document search finds a passage, while text-to-SQL calculates a number, and a wrong filter gives a confident, wrong number.

How often the answer is right

Public benchmarks give a sober picture. BIRD, a benchmark of 95 databases, counts an answer as correct when the result of the generated query matches that of a reference query. On 30 October 2026, human performance on its test set is 92.96 per cent. The best single model, Google's Gemini-SQL2, reaches 80.04 and is listed without a link to weights; the best one with a link to weights on Hugging Face, Kwai-AutoSQL-32B, reaches 74.12. The top entry on the main leaderboard, which ranks complete systems, stands at 83.40 (entry dated 22 October 2026).

With the best open-weight model on the leaderboard, roughly one answer in four is wrong on a public test, and company databases are messier than BIRD. Three habits make the tool usable all the same. Show the SQL with every answer, so that someone who knows the data can spot a wrong join. Keep a library of checked questions with their correct queries, which the tool reuses as examples. And mark answers that came from a checked query differently from new ones. Spider 2.0, a harder benchmark with enterprise databases that often have more than 1,000 columns, adds a second lesson: on its Snowflake set, gpt-oss-120b scored 65.45 in one agent framework and 13.10 in another, as our text-to-SQL guide records, so the system around the model matters as much as the model.

Open tools for chat with your database in October 2026

Several open projects promise exactly this. Their status changed during 2026, so check it before you build on one.

GitHub repositories and README files of Vanna, WrenAI, DB-GPT and LangChain, DB-GPT release page, LangChain's "Build a SQL agent" tutorial and ChatOllama page, all read on 30 October 2026.

Vanna calls itself "Chat with your SQL database" and was rewritten as Vanna 2.0 with user-aware agents and a ready-made chat component. Its repository was "archived by the owner on Mar 29, 2026" and "is now read-only", so it receives no further fixes. We do not install it for new projects.

WrenAI describes itself as "GenBI (Generative BI) for AI agents, an open-source, governed text-to-SQL through an open context layer". Its core engine, SDK and skills are under Apache 2.0; row- and column-level security with user management, the GenBI user interface, dashboards and embedding are listed as commercial. Its README lists coding agents as the clients of the open part and names no local model provider. Its modelling language serves as the example of a semantic layer below.

DB-GPT, MIT-licensed, calls itself an "open-source agentic AI data assistant". It lets users ask questions in natural language, writes the SQL, and generates charts, dashboards and reports. Its README names local GPU models through vLLM and llama.cpp and OpenAI-compatible model servers, and opens a web interface on port 5670. It says nothing about user accounts or permissions, so access belongs in front of it, through the company sign-in or an authenticating proxy, as our guide to a private ChatGPT alternative describes for chat front ends.

LangChain's SQL agent is a tutorial with building blocks for developers; there is no finished app to install. The agent lists the tables, reads the structure of the relevant ones, writes a query, has the model check it, runs it, corrects it after an error and answers. Its system prompt says "DO NOT make any DML statements (INSERT, UPDATE, DELETE, DROP etc.)", and a HumanInTheLoopMiddleware can pause before every query until a person approves it. The tutorial's examples use cloud models; LangChain's ChatOllama integration supports tool calling with a local model.

You can order the RTX PRO workstation alone, the setup of the model, DB-GPT and the read-only database access alone, or both together: the workstation delivered with everything installed and configured; write to us and we suggest the best solution for your tasks and budget.

A semantic layer with your business terms

The model sees table and column names, and even clear names do not tell it what your company means by "revenue", which order statuses count as cancelled, or which date decides the quarter. A semantic layer is a written description of the data in business terms that sits between the database and the model: what each table means, which joins are approved, how each key figure is calculated.

WrenAI's context layer shows what such a layer contains: its Modeling Definition Language describes models, columns, relationships, views, cubes and metrics, and two plain files, instructions.md and queries.yml, hold enums, units, approved joins and definitions. The same idea works without WrenAI. The IT team builds a handful of database views, saved queries that look like tables, with readable names for the questions that matter, such as sales by customer and month or open orders with sales rep. It writes one sentence per column and adds the checked example questions. The model then queries only these views, and the raw ERP tables stay out of its reach.

This step decides more than the model choice. A controller who writes down twenty definitions and thirty checked questions gives the tool what an experienced colleague knows. Business definitions also change; keep the layer in Git so that every change is dated and can be reviewed.

Read-only access to ERP and CRM data

The database account that the tool uses is the real safety boundary. LangChain's documentation puts it plainly: "Make sure that your database connection permissions are always scoped as narrowly as possible for your agent's needs." Its instruction not to change data is a sentence in a prompt; only the database rights enforce it.

1. Query a copy, not the live system. Use a reporting replica or the data warehouse, so that a heavy query does not slow down order entry. Ask your ERP provider whether direct read access to its database is allowed under your support contract.
2. Create a read-only user. Give it SELECT rights on the views of the semantic layer and nothing else. In PostgreSQL, default_transaction_read_only is only a default that a session can change, so the rights decide; statement_timeout stops runaway queries and is set for this account, as our text-to-SQL guide explains.
3. Keep row-level rules. If sales reps may only see their own customers, that rule belongs in the database, and the database has to know who is asking: one shared account sees everything its role may see. Give each team its own read-only role and its own connection in the tool. In PostgreSQL, create the views with security_invoker = true, because by default a view applies the row-level rules of its owner, not of the person asking.
4. Check and log every query. Accept one SELECT statement per question, add a row limit, and log the question, the SQL and the user.

Customer notes and free-text fields can contain instructions that end up in the model's context, a risk described in our guide to prompt injection and LLM security. Personal data in the CRM stays subject to the GDPR on your own hardware as well; what own hosting settles and what remains your duty is set out in our guide to local AI and the GDPR, and your data-protection officer confirms the details.

Which local model writes the SQL

Two kinds of model are used. Models fine-tuned for SQL, such as Kwai-AutoSQL-32B (weights under Apache 2.0) or Snowflake's Arctic-Text2SQL-R1-7B (Apache 2.0, 68.5 on the BIRD test set by its card), turn a question and the table definitions into one query. Agents such as DB-GPT or the LangChain SQL agent need a general model that calls tools: gpt-oss-20b or gpt-oss-120b (Apache 2.0), or Qwen3.8-27B (Apache 2.0, August 2026), whose card lists no SQL benchmark. No published figure compares these general models on your kind of schema, so a test with your own questions decides.

Qwen3.8-27B thinks before it answers, and its card sets the highest reasoning effort as default and offers medium and low; for SQL steps a lower effort shortens the wait. Speed for one question depends on memory bandwidth and the bytes read per token: on the RTX PRO 5000 (1,344 GB/s) the 8-bit 27B model reads about 28 GB per token, so about 47 tokens per second is an upper bound by our estimate. An agent that writes 500 tokens in each of four steps therefore needs at least about 43 seconds, also by our estimate.

Cards and capacity for database questions

We size with our users-per-card rule: 90 per cent of the memory the driver reports, less 3 GiB per card, less the model weights, leaves memory for the questions in progress. A token is a word or part of a word. One question with instructions, the selected views, examples and the agent's steps is about 10,000 tokens in this example, the same assumption as in our text-to-SQL guide. We size with a 16-bit cache, as Ollama keeps it by default.

Our estimates by the users-per-card rule (usable memory 18.5, 40.0, 61.5 and 83.0 GiB); weights from Ollama's tags (qwen3.8 q8_0 30 GB) and the Hugging Face files of gpt-oss; 16-bit cache per token: 24 KiB for gpt-oss-20b, 64 KiB plus 0.14 GiB of fixed state for Qwen3.8-27B, 36 KiB for gpt-oss-120b. An FP8 cache, as vLLM offers it, leaves room for more questions. Not measurements.

Questions in progress are not users: ten managers rarely ask in the same second, so each count serves a larger team. The 24 GB entry card suits a team with a few clean views. The 48 GB RTX PRO 5000 holds a 27B model at 8-bit with room for schema and examples; our page on the RTX PRO 5000 48 GB workstation covers the machine. The 72 GB version serves several departments, and one RTX PRO 6000 Workstation Edition or Max-Q runs gpt-oss-120b. For a BI team on a large warehouse, a server with RTX PRO 6000 Server Edition or H200 NVL cards is sized in our text-to-SQL guide.

Send us your database type, the number of tables and ten questions your team asks, and we suggest the card and the setup that fit.

Starting with a pilot

Collect 50 to 100 real questions from the people who will use the tool, with the correct answer checked by someone who knows the data. Build the views and definitions for them, run the questions through each candidate model with the same tool, and count correct results, wrong results and refusals. Repeat the test after every change to the model, the prompt or the views; our guide to LLM evaluation shows how to keep such a test set. A first pilot on one subject area, such as sales or purchasing, shows whether the answers can be trusted before more areas are added. When more departments want it, the step from one workstation to a shared machine is described in from Ollama on a PC to a company AI server.

FAQ