Text-to-SQL on-premise: open models, schema context, read-only access and the GPU server

What text-to-SQL on your own server does

Text-to-SQL on-premise lets staff ask questions of internal databases in plain language while the language model, the schema and the results stay on your own server. An open model on one NVIDIA RTX PRO 6000 Blackwell Server Edition serves a BI team: on the public BIRD benchmark, open-weight models of 7B to 32B parameters reach about 68 to 74 per cent execution accuracy on the test set, and enterprise-scale tasks need an agent that explores the schema, runs and corrects queries. The database account the model uses decides what can go wrong; it should only be able to read.

A request runs in five steps. The system picks the tables and columns that matter for the question, the model writes a query, a validator checks that it is a single read statement, the database runs it on a read-only account, and the user sees the result together with the SQL. The last point matters for trust: an analyst can read the query and spot a wrong join, which a chart alone does not reveal.

This is a different problem from retrieval over documents. Our guide to RAG on company data finds passages in text; text-to-SQL computes an answer from rows, so a wrong filter gives a confident, wrong number. That is why model choice, schema context and query checks get their own sections here, and why a test set of your own questions comes before the hardware order.

Public leaderboards: BIRD and Spider 2.0 in October 2026

BIRD measures execution accuracy: a generated query counts as correct when its result matches that of the reference query. Its benchmark has 95 databases totalling 33.4 GB, more than 12,751 question and SQL pairs and more than 37 professional domains. Its single-model track ranks one trained model without an agent framework around it.

BIRD leaderboard, single trained model track, bird-bench.github.io, read on 24 October 2026. Human performance 92.96. The top single model, Google's Gemini-SQL2, reaches 80.04 and is not an open model; the top system entry overall reaches 82.95. Arctic-Text2SQL-R1-7B is not listed on this track; its own card reports 68.5 on the BIRD test set.

Kwai-AutoSQL-32B is a fine-tune of Qwen3-32B. Its card states that weights and configuration are Apache 2.0 derivatives of Qwen3-32B, with the README and root licence file under MIT; it gives launch commands for vLLM and SGLang and no benchmark, prompt format or dialect. Arctic-Text2SQL-R1-7B, from Snowflake, is a fine-tune of Qwen2.5-Coder-7B-Instruct under Apache 2.0; its card reports 88.8 on Spider test and 15.6 on Spider 2.0-DK. Both cards are silent on which SQL dialects they were trained for. BIRD's main set is SQLite; its Mini-Dev set of 500 examples covers SQLite, MySQL and PostgreSQL, and there one entry built on gpt-oss-120b, FrontierSQL, reaches 56.80 on PostgreSQL.

Spider 2.0 is closer to a company warehouse. Its Snow setting has 547 tasks on Snowflake, its Lite setting 547 across BigQuery, Snowflake and SQLite, and the project states that its databases often have more than 1,000 columns. The leaders on Spider 2.0-Snow are agent systems, at 96.70. Open models appear lower and spread widely: DeepSeek-V3.2 in the QiSi-SQL framework 70.38, gpt-oss-120b in FlexSQL 65.45 and in AI-DIVE 13.10. On Spider 2.0-Lite, AI-DIVE scored 42.40 with DeepSeek-V4-Flash and 21.90 with gpt-oss-120b. The same model can sit at either end of the table, and the schema linking, retries and checks around it make the difference. Our guide to choosing an open LLM for a company assistant covers licences and languages for the general-purpose models in these entries.

Schema size is the context budget

The model can only write a join it knows about, so the schema goes into the prompt. A rough rule is tables × columns × tokens per column. With compact table definitions, a column name, its type and a short comment take about 10 to 15 tokens, our assumption; count with the model's tokenizer on your own definitions before you plan.

Our estimate at 10 to 15 tokens per column. Context limits: max_position_embeddings in the config.json of Kwai-AutoSQL-32B (40,960) and Arctic-Text2SQL-R1-7B (32,768), read on 24 October 2026.

Schema retrieval solves the large case. Describe each table and its important columns in a sentence or two, embed those descriptions, and select the 10 to 20 tables closest to the question before the model writes SQL; foreign keys from the catalogue add the join tables. The embedding and reranker models for this step are small, as our guide to an embedding and reranker server for RAG shows. Business definitions belong in the same retrieved context: what "revenue" means in your company, which status codes count as cancelled, and a few verified example queries. A query can be valid SQL and still use the wrong definition.

A fixed part of the prompt, the instructions and the core tables, is the same for every request. Serving engines can reuse its cache across requests, which shortens the time to the first token, as our guide to AI agents and GPU sizing explains for prefix caching.

Read-only access and query checks

The database account the model uses is the security boundary; an instruction in the prompt stops no query. Give it SELECT on the views or schemas it may read, nothing else, and run it on a read replica or a reporting copy where a heavy query cannot slow the production system. If you use PostgreSQL 18's predefined role pg_read_all_data instead of grants per view, it reads all tables, views and sequences, and its documentation states that it "does not bypass row-level security (RLS) policies", so row-level rules for departments or clients keep applying to the model's queries.

Two settings help and do not replace the grants. default_transaction_read_only sets the default read-only status of each new transaction; the documentation describes later changes as equivalent to a SET TRANSACTION command, so a session can change it, and the grants still decide. statement_timeout aborts any statement that runs longer than the set time; the documentation advises against setting it in postgresql.conf because it would affect all sessions, so set it for the text-to-SQL account.

On the application side, parse every generated statement and accept exactly one SELECT, add a row limit, log the question, the SQL and the user, and show the SQL with the answer. Data values can carry instructions too: a customer note that says "ignore the filter" ends up in the model's context when the agent reads sample rows, which our guide to prompt injection and LLM security covers. We supply the servers and, with our engineering partner Vixen.UNO, set up the model, schema retrieval and query checks; send us your database type, schema size and the questions your team asks, and we reply with a configuration.

Multi-step agents and their latency

For enterprise schemas, the better results come from agents: the model lists candidate tables, inspects columns and sample values, writes a query, runs it, reads the error or an empty result and corrects itself. Each step is a model call with tool use, so a question takes several calls; our guide to vLLM tool calling covers the parsers that turn the model's output into tool calls.

Generation time then adds up. For a dense model, each output token reads all weights once, so memory bandwidth divided by the weight size gives a ceiling per request. Kwai-AutoSQL-32B in BF16 reads 65.5 GB per token: about 24 tokens per second on an RTX PRO 6000 Server Edition at 1,597 GB/s and about 73 on an H200 NVL at 4.8 TB/s, an upper bound by our estimate. An agent that writes 1,000 tokens of reasoning and SQL in each of four calls waits at least about 164 seconds on the first card and 55 on the second. Quantised to FP8, one byte per parameter, the model reads about half as much, and our guide to quantising an LLM to NVFP4 or FP8 covers the tools and the accuracy check that has to follow.

A mixture-of-experts model reads only its active experts. gpt-oss-120b reads about 5 GB per token, which puts its ceiling on one Server Edition card at about 319 tokens per second, an upper bound by our estimate because batched tokens hit different experts. StorageReview measured 176.09 tokens per second for one user of gpt-oss-120b in NVFP4 on two Server Edition cards with vLLM. That makes it a reasonable agent model when a fine-tuned 32B is too slow in BF16, provided it passes your own test set.

Sizing for a BI team: RTX PRO 6000 Server Edition or H200 NVL

We size with our users-per-card rule: 90 per cent of the memory the driver reports, less 3 GiB per card, less the weights, leaves the KV cache. That gives 83.0 GiB per RTX PRO 6000 Server Edition and 123.4 GiB per H200 NVL. A text-to-SQL request with a retrieved schema, examples and the model's reasoning is about 10,000 tokens in this example.

Our estimates; SE is the RTX PRO 6000 Server Edition, Kwai 32B is Kwai-AutoSQL-32B in BF16, Arctic 7B is Arctic-Text2SQL-R1-7B. Weights: safetensors files on Hugging Face (Kwai-AutoSQL-32B 65.5 GB, Arctic-Text2SQL-R1-7B 15.2 GB, gpt-oss-120b 65.3 GB). Cache per token in 16-bit from config.json: 256 KiB for the 32B (64 layers, 8 KV heads of 128), 56 KiB for the 7B (28 layers, 4 KV heads of 128), 36 KiB for gpt-oss-120b. Ceilings are bandwidth ÷ bytes read per token, upper bounds; the TP=2 ceiling ignores the exchange between the cards.

For a BI team of 20 to 50 analysts we assume a few questions in flight at the same moment, so one RTX PRO 6000 Server Edition carries the 32B model by memory. The limit is waiting time per question: with an agent loop in BF16, the H200 NVL's bandwidth roughly triples generation speed, and two Server Edition cards in a tensor-parallel pair double it at best. A 7B fine-tune on one Server Edition card is fast and leaves room for a second model, at about five and a half points less on the BIRD test set (68.5 on the Arctic 7B card against 74.12 for Kwai-AutoSQL-32B on the leaderboard). Our RTX PRO 6000 vs H200 NVL comparison covers the cards in general; the entry server on our AI servers page has two RTX PRO 6000 Server Edition cards and draws about 1.8 kW at the wall by our estimate, and our page on a server with two H200 NVL covers the HBM option.

Testing on your own questions

Collect 100 to 200 real questions from the analysts, with the correct SQL written or checked by someone who knows the data. Score them the way BIRD does, by comparing result sets, and break the score down by type: simple lookups, joins across three or more tables, time windows, and questions that depend on a business definition. Run every candidate model with the same schema retrieval and the same checks, in the dialect of your database, and measure the time per question with the agent loop switched on. Keep the set out of any fine-tuning and run it again after each change of model, prompt or schema description.

Our Private AI/ML service sets up such a pilot on your data; describe your databases, the users and the questions they ask, and our team proposes the test and sizes the server from its results.

What we supply

You can buy this system from us: our team sizes the configuration to your workload and budget and replies within one business day. We supply the RTX PRO 6000 Server Edition and the H200 NVL as GPUs and in AI servers built to order, assembled and tested before delivery, on one EU contract and invoice with manufacturer warranty on every component. Operating system, drivers, CUDA and a container runtime are installed on request; the model, schema retrieval and query checks are part of our Private AI/ML service.

FAQ