AI test generation on-premise for unit, integration and Playwright tests with a local LLM

Tests written by a model on your own server

A QA or development team can generate unit, integration and end-to-end tests with an open model on its own GPU, so the source code goes only to the company's own model server. A coding agent writes a test, runs it, reads the failure and fixes it; Playwright's Test Agents do the same for browser tests, and mutation testing then checks whether the new tests catch faults. One NVIDIA RTX PRO 6000 Blackwell Workstation Edition or Max-Q with 96 GB serves a team, and a server with RTX PRO 6000 Server Edition cards runs the same jobs for a whole department in the nightly pipeline.

This guide is for QA leads, test automation engineers and heads of development. It summarises what research has measured, which tools write which kind of test, how to check the output, and which hardware fits. How the agents connect to a local model is set out in our guide to Claude Code with a local model; embedded C and MISRA have their own guide, local AI for embedded C unit tests, and review bots for merge requests are covered in self-hosted AI code review.

What research has measured on LLM-written tests

arXiv abstracts of 2402.09171 (14 February 2024), 2501.12862 (22 January 2025), 2403.16218 (v4, 9 May 2025) and 2410.00752 (v2, 18 March 2025), read 30 October 2026. Only TestGenEval names a model in its abstract; none of the studies reports results on RTX PRO hardware.

Meta's TestGen-LLM paper, "Automated Unit Test Improvement using Large Language Models at Meta", describes a tool that improves existing human-written tests and checks each candidate against filters before a person sees it. In its Instagram evaluation, 75 per cent of the generated test cases built, 57 per cent passed reliably and 25 per cent increased coverage. By those figures, three in four generated tests added no coverage, and the filters removed them before review.

The follow-up paper "Mutation-Guided LLM-based Test Generation at Meta" turns the order round. Its ACH system first generates a few mutants, simulated faults tied to one concern such as privacy, and then asks the model for tests that catch them. Across 10,795 Android Kotlin classes on seven platforms it produced 9,095 mutants and 571 privacy-hardening tests, and engineers at Messenger and WhatsApp test-a-thons accepted 73 per cent of its tests.

Two academic benchmarks give very different numbers. CoverUp, a coverage-guided loop for Python, reached a median per-module line and branch coverage of 80 per cent against 47 per cent for CodaMosa. TestGenEval asked models to write complete test files for 1,210 code files from 11 Python repositories, and the best model in the paper, GPT-4o, reached "an average coverage of only 35.2%". The two studies use different projects, models and tasks, so the figures cannot be compared directly. CoverUp's authors credit their result both to the model and to the way coverage analysis, code context and feedback are combined over several rounds, which is the same principle as Meta's filters.

Tools for unit, integration and end-to-end tests

Playwright documentation and release notes, the Qodo Cover repository and the Playwright repository, read 30 October 2026; OpenCode and Claude Code as verified in our Claude Code guide on 30 October 2026; Anthropic's gateway documentation re-read on 30 October 2026.

Coding agents for unit and integration tests. OpenCode and Claude Code read the repository, write a test file, run the test command in the terminal and repair what fails. OpenCode connects to an OpenAI-compatible endpoint such as vLLM or Ollama. Claude Code reaches a local model through the Anthropic-format endpoint that both serve, but Anthropic's documentation states that it "doesn't support routing Claude Code to non-Claude models through any gateway", so that setup runs outside its support. Give the agent a project instruction file with the test framework, the naming rules, the fixtures to reuse and the command that measures coverage.

Playwright Test Agents for end-to-end tests. Playwright 1.56 introduced "three custom agent definitions designed to guide LLMs": the planner explores the application and writes a Markdown test plan, the generator turns the plan into Playwright test files and checks selectors and assertions against the running application, and the healer runs the suite and repairs failing tests. npx playwright init-agents writes the definitions into the repository, with loops for VS Code, Claude Code and OpenCode in the 1.56 release notes; the documentation says "use your AI tool of choice" and asks for the definitions to be regenerated whenever Playwright is updated. Plans go to specs/, tests to tests/, starting from a seed test that provides a ready-to-use page context. The documentation notes that the healer can also leave a test skipped if it judges the functionality broken, so a reviewer looks at every skip. Since version 1.62 Playwright also bundles the Playwright MCP server.

Qodo Cover, formerly a popular open tool for coverage-raising unit tests, has stated since 15 June 2025: "This repository is no longer maintained. Please fork it if you wish to continue development or use it in your own projects." We do not install it.

You can order the workstation or server alone, the setup of vLLM, a coding model, OpenCode and Playwright's agents alone, or both together: the system delivered with everything installed and configured; write to us and we suggest the best solution for your tasks and budget.

Mutation testing as the quality gate

Coverage shows which lines a test executes. Whether the test would notice a bug is a separate question: a generated test that calls a function and asserts nothing raises coverage and catches nothing. Mutation testing measures that second property. In PIT's words, "Faults (or mutations) are automatically seeded into your code, then your tests are run. If your tests fail then the mutation is killed, if your tests pass then the mutation lived." A surviving mutant points to an assertion the tests lack.

pitest.org and the PIT, Stryker and mutmut repositories, read 30 October 2026. PIT works with Ant, Maven and Gradle; mutmut needs a system with fork support, so it runs on Windows only under WSL.

A gate for generated tests follows Meta's filters, with mutants added:

1. Builds. The test compiles and its imports resolve.
2. Passes repeatedly. It passes several runs in a row, which removes flaky tests that depend on timing or order.
3. Adds coverage. Coverage on the changed code rises against the existing suite.
4. Kills mutants. The mutation score on the touched code goes up, or surviving mutants are fed back to the agent as the next task, as ACH does.
5. Passes review. A developer reads the assertions before merge. A test that freezes a current bug into an expected value passes every automatic check.

Mutation testing runs the tests again for every mutant, on the processors of your build machines and not on the GPU, so restrict it to the changed files in each merge request and run the full analysis at night.

Which model, and how much context

A test-writing agent sends the instructions, the code under test, existing tests and every test run's output on each step, so it needs a model with reliable tool calling and a context of at least 64,000 tokens, the minimum Ollama asks for coding agents. We size at 128K tokens, as in our Claude Code guide.

The open models that fit one or two workstation cards are Qwen3.6-35B-A3B (35B total, 3B active, SWE-bench Verified 73.4 by Qwen), the dense Qwen3.8-27B (SWE-bench Pro 61.7 in the Claude Code harness, by Qwen) and Qwen3-Coder-Next (80B total, 3B active), all under Apache 2.0. Their cards report repository-repair and terminal benchmarks; neither the Qwen3.6-35B-A3B card nor the Qwen3.8-27B card reports a test-generation benchmark. Run a pilot on your own code instead: pick 20 classes without tests, let the agent work, and count the tests that pass the gate above. Qwen3.6-35B-A3B computes about 3B of its 35B parameters per token, so it reads only a fraction of the weights the dense Qwen3.8-27B reads and, by our arithmetic, answers several times faster, which matters when an agent iterates dozens of times per file.

GPUs from one developer to the nightly pipeline

Interactive work needs fast answers for each developer; nightly generation needs throughput. Our users-per-card rule takes 90 per cent of the memory the driver reports, less 3 GiB per card, less the weights, as explained in our guide to how many users one RTX PRO 6000 serves.

sessions held in memory by our users-per-card rule with a 16-bit cache (FP8 cache in brackets), from our Claude Code guide of 30 October 2026; for interactive work that guide plans fewer sessions at 50 tokens per second, for example 11 (23) on one RTX PRO 6000 Workstation Edition or Max-Q, an upper bound by our estimate.

Entry. One RTX PRO 4500 Blackwell (32 GB) serves one developer who writes tests with an agent. The RTX PRO 5000 Blackwell with 72 GB serves a small QA team.

Mid. One RTX PRO 6000 Workstation Edition or Max-Q (96 GB) holds 18 sessions of the 35B-A3B model and suits a team of about ten; 11 of them run at the planned 50 tokens per second, an upper bound by our estimate for this mixture-of-experts model. For those who prefer the dense model, the same card holds Qwen3.8-27B in FP8 with 6 sessions of 128K in memory, 13 with an FP8 cache, at a ceiling of about 63 tokens per second for one request by our estimate. Two cards run Qwen3-Coder-Next as a tensor-parallel pair. Two Workstation Edition cards at 600 W each are the realistic maximum for that edition; three or four Max-Q cards belong only in a chassis built for them with strong directed airflow.

Large. For nightly generation across many repositories a rack server with RTX PRO 6000 Server Edition cards takes over. The nearest published throughput figure is NVIDIA's TensorRT-LLM result for Qwen3-30B-A3B in FP4 on one RTX PRO 6000 Server Edition: 374 output tokens per second with 32,768-token prompts and 1,024-token answers at maximum offline throughput. By our arithmetic that is 374 × 60 ÷ 1,024, about 22 calls a minute. If an agent needs 20 calls to finish the tests for one module, an assumption for illustration, one card completes at most about 66 modules an hour, an upper bound by our estimate. Count the calls per module in your pilot. Four cards at 600 W with two AMD EPYC 9355 processors draw about 3.4 kW by our estimate. Our guide to a server for coding agents sizes the shared case.

Send us the number of developers, the languages and test frameworks, and how many repositories the nightly run should cover; we size the workstation or server and the model for that load.

Test generation in the CI pipeline

The GPU machine serves the model; the tests run where they always run. A CI job checks out the branch, starts the agent with the project's instruction file, lets it write and run tests in the job's container, applies the gate and opens a merge request with the tests that passed. Browser tests need browsers and the application under test in that container, which costs processor time and memory on the runners, not on the GPU server. Our guide to GPU CI runners covers runners that need a GPU themselves.

Four settings reduce the risk. Run the agent in a container without production credentials, because it executes the code it writes. Give each pipeline its own key on an LLM gateway with keys and logging, since vLLM's own --api-key protects only some paths and manages no users. Set the context the server was started with in the agent's configuration, so long test runs are compacted before the server rejects a request. And set "share": "disabled" in OpenCode's configuration, because its /share command uploads the conversation, which can contain code, to opencode.ai.

FAQ