Claude Code, OpenCode, Codex and Copilot CLI with a local model: setup, models and GPUs

Which terminal coding agents run on a local model

Claude Code, OpenCode, Codex CLI and GitHub Copilot CLI can all work against a model on your own GPU instead of a cloud API. Claude Code connects to an Anthropic-format endpoint that Ollama and vLLM provide, the other three usually to an OpenAI-compatible one, and every one of them needs a model with tool calling and a context of at least 64,000 tokens. For one developer, a workstation with one RTX PRO 4500 or RTX PRO 5000 is the start; one or two NVIDIA RTX PRO 6000 Blackwell Workstation Edition or Max-Q cards carry a small team, and a server with RTX PRO 6000 Server Edition cards takes over for a department.

This guide covers the terminal agents and the step from one workstation to a shared server. The IDE extensions, completion models and licences of the coding models are in our guide to a private coding assistant, the sizing of servers for 20 to 100 developers in our guide to a server for coding agents, the IDE tools that replace GitHub Copilot in our comparison of self-hosted Copilot alternatives, and the Cursor editor in our guide to Cursor with a local model.

Licence fields from the npm registry and the projects' repositories; connection methods from Anthropic's Claude Code, Ollama, OpenCode, OpenAI Codex and GitHub documentation; all read on 30 October 2026. All four clients receive frequent releases, so we give no version numbers except where a feature needs a minimum version.

Claude Code and Copilot CLI are proprietary clients. Anthropic's legal page says that use of Claude Code is subject to its Commercial Terms of Service for Team, Enterprise and API users and to its Consumer Terms for Free, Pro and Max users, and its gateway documentation states that Anthropic "doesn't support routing Claude Code to non-Claude models through any gateway". The setup below works technically, as Ollama documents, but runs outside Anthropic's support. The Copilot CLI README lists an active Copilot subscription as a prerequisite; GitHub's changelog of 7 April 2026 adds that "GitHub authentication is not required" when you use your own model provider. Your legal counsel checks the terms before a rollout. Our setup covers the model server and the model; your developers install Claude Code or Copilot CLI themselves, under Anthropic's or GitHub's terms.

Setting up Claude Code and the other agents with Ollama

Ollama is the shortest route for one developer. Since version 0.15, ollama launch starts Claude Code, OpenCode, Codex or Droid against a local model with one command, and Ollama's blog post of 23 January 2026 asks for a context length of at least 64,000 tokens. Use Ollama 0.35.0 or later: earlier versions are affected by CVE-2026-103663, a path traversal in /api/pull, as our guide to moving from Ollama on a PC to a company AI server explains. Ollama is actively developed as of October 2026, so keep it updated.

1. Claude Code. Ollama's guide sets ANTHROPIC_BASE_URL to http://localhost:11434, ANTHROPIC_AUTH_TOKEN to ollama ("required but ignored") and ANTHROPIC_API_KEY to an empty string, then starts claude --model with a local model.
2. OpenCode. An Ollama provider in opencode.json with the @ai-sdk/openai-compatible package and baseURL set to http://localhost:11434/v1. OpenCode's own documentation shows the same pattern for LM Studio and llama-server.
3. Codex CLI. codex --oss runs against Ollama or LM Studio, for example codex --oss -m gpt-oss:120b; the Codex profile in Ollama's guide sets wire_api to the Responses API.
4. GitHub Copilot CLI. COPILOT_PROVIDER_BASE_URL points at the endpoint, COPILOT_MODEL names the model, and COPILOT_OFFLINE=true stops the CLI from contacting GitHub's servers. GitHub's documentation lists Ollama and vLLM as providers and returns an error if the model lacks tool calling or streaming.

Claude Code needs three more settings that the quick guides leave out. It does not recognise a local model name, so it assumes a context window of 200K tokens and compacts the conversation only there; Anthropic's model configuration page says to set CLAUDE_CODE_MAX_CONTEXT_TOKENS to the real window, which is the context the server was started with. Background tasks use the model behind the haiku alias and subagents use CLAUDE_CODE_SUBAGENT_MODEL, so point ANTHROPIC_DEFAULT_HAIKU_MODEL and that variable at the local model as well; otherwise those requests name Claude models that your server does not serve.

Ollama's Anthropic layer lists what it leaves out: prompt caching, the token counting endpoint, tool_choice and PDF blocks. Claude Code then counts context through the messages endpoint, as Anthropic's gateway reference describes, and the cache markers it sets on the system prompt have no effect. Keep Ollama on 127.0.0.1, its default; its container image listens on all interfaces instead.

You can order the workstation alone, the setup of Ollama or vLLM and the coding model alone, or both together: the workstation delivered with everything installed and configured; write to us and we suggest the best solution for your tasks and budget.

What still leaves the machine

Pointing an agent at a local model keeps prompts and code on your network, but the clients have their own traffic. Our article on telemetry in self-hosted AI tools covers the server side; for the four clients, their vendors document the following, as read on 30 October 2026.

• Claude Code keeps its usage metrics on by default except on the cloud providers Anthropic lists, and according to Anthropic they never include code, prompts or file paths. CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC set to 1 disables auto-updates, telemetry, error reporting, the /feedback command and release notes. The WebFetch tool still sends the hostname of each requested URL, without path or content, to api.anthropic.com for a safety check until skipWebFetchPreflight is set to true in the settings.
• Copilot CLI in offline mode disables telemetry and talks only to the configured provider, per GitHub's changelog; GitHub's documentation adds that full network isolation holds only if the provider is local too.
• Codex CLI "periodically sends a small amount of anonymous usage and health data back to OpenAI" by default; [analytics] enabled = false turns it off, and [feedback] has its own switch. Under ollama launch, Ollama executes Codex's web searches, which need ollama signin; Ollama's guide shows how to disable them per session.
• OpenCode uploads a conversation to opencode.ai only when someone runs /share, since automatic sharing is off by default; "share": "disabled" in opencode.json turns sharing off entirely.
• Ollama offers cloud models next to local ones; OLLAMA_NO_CLOUD=1 removes them, as our telemetry article sets out.

A firewall that denies outbound traffic from developer machines except to the model server and the package mirror catches what a setting misses.

Which open model holds up as the backend

A terminal agent sends its whole task on every step: instructions, tool definitions, the files it has read and the output of each command. The model has to call tools in the expected format, follow long instructions and stay consistent over 50 or 100 steps. The cards below are the vendors' own figures; SWE-bench Verified measures how many real repository issues an agent resolves.

Hugging Face model cards, configuration files and file listings, read on 30 October 2026 (Qwen3.6, Qwen3.8, Qwen3-Coder-Next, GLM-4.7-Flash) and in our sibling articles (gpt-oss, GLM sizes, Qwen3.8 FP8 size). Scores are vendor-reported; gpt-oss-20b's figure comes from the comparison table on the GLM-4.7-Flash card. The Qwen3.8-27B card reports SWE-bench Pro, not Verified. Cache per session in 16-bit by our arithmetic from each config.json, including the fixed state of linear-attention layers; an FP8 cache halves the per-token part. gpt-oss-20b and GLM-4.7-Flash are shown at 64K.

All seven are Apache 2.0, except GLM-4.7-Flash, which is MIT. Qwen3.8-27B, which Qwen published in August 2026, is the newest of them and the model our vibe coding workstation guide starts from. Its card is the one most relevant here: Qwen ran SWE-bench Pro for its own models in the Claude Code harness, with a 256K context, and reports 61.7 for Qwen3.8-27B against 53.5 for Qwen3.6-27B; the 53.4 it lists for Claude Opus 4.6 Max is the officially reported score, so that pair is not like-for-like. Three differences matter in practice.

Speed. A mixture-of-experts model reads only its active parameters per token. Qwen3.6-35B-A3B computes about 3B of its 35B parameters per token, while the dense Qwen3.6-27B reads all of its weights for every token, so by our arithmetic the MoE model answers several times faster. The dense model scores higher, and an agent feels the difference in speed on every step. No single-request measurement of the 8-bit dense models on RTX PRO cards is published. Qwen3.8-27B has the same layers and attention heads as Qwen3.6-27B and 8-bit weights of the same size, and the speed of one request depends mainly on the bytes read per token, about 28 GB: by our estimate that caps one request at about 63 tokens per second on a Workstation Edition or Max-Q, 56 on the Server Edition and 47 on an RTX PRO 5000. Real speeds are lower: StorageReview's test of Qwen3.8-27B in NVFP4, described in our vibe coding workstation guide, measured 17 to 22 tokens per second for one request on the cards it tested.

Cache per session. The Qwen3.6 and Qwen3.8 models and Qwen3-Coder-Next keep a per-token cache in only one layer in four. A 128K session of Qwen3.6-35B-A3B or Qwen3-Coder-Next costs 2.6 to 3.1 GiB, the dense 27B models with more attention heads about 8.1 GiB, and the older Qwen3-Coder-30B-A3B 12 GiB. The small cache is what lets one card hold many long sessions.

Thinking. The Qwen3.6 and Qwen3.8 cards state that the models think by default, which adds output tokens to every step; Qwen3-Coder-Next supports only non-thinking mode. Qwen's FP8 card for Qwen3.6-35B-A3B recommends keeping at least 128K of context to preserve thinking, one reason we size at 128K.

The large open coders, such as Kimi K2.7-Code, need an eight-card server; our page on Kimi K2.7-Code on 8× RTX PRO 6000 Server Edition has that configuration.

Which GPU for one developer, a team or a department

Our users-per-card rule takes 90 per cent of the memory the driver reports, less 3 GiB per card, less the weights, as in our guide to how many users one RTX PRO 6000 serves: 25.7 GiB on an RTX PRO 4500, 61.5 GiB on an RTX PRO 5000 72 GB and 83.0 GiB on an RTX PRO 6000 of any edition. Speed is the second limit. Each decoding step reads the active weights and the cache of every session, so we also count the sessions that keep a ceiling of 50 tokens per second, the method of our coding-agent server guide. For mixture-of-experts models that ceiling is an upper bound by our estimate, because batched tokens reach different experts. We count one generating session per developer at the busy hour.

Sessions of 128K tokens by our users-per-card rule, 16-bit cache (FP8 cache in brackets); planned is the smaller of memory and our 50 tokens-per-second ceiling, an upper bound by our estimate. Bandwidth from NVIDIA: 896 GB/s (RTX PRO 4500), 1,344 GB/s (RTX PRO 5000), 1,792 GB/s (Workstation Edition and Max-Q), 1,597 GB/s (Server Edition); active weights about 3 GB per token. Server rows from our coding-agent server guide.

Entry: one developer. A workstation with one RTX PRO 4500 Blackwell (32 GB) holds NVIDIA's NVFP4 build of Qwen3.6-35B-A3B with one 128K session, two with an FP8 cache. Ollama states about 23 GB of VRAM for GLM-4.7-Flash at 64,000 tokens, which also fits this card. A 24 GB RTX PRO 4000 leaves 18.5 GiB by our rule, enough for gpt-oss-20b with three 64K sessions; in GLM's comparison table that model resolves about a third of the SWE-bench Verified issues, so it suits small edits. Our article on 30B mixture-of-experts models on 24 GB cards has the detail. The RTX PRO 5000 72 GB holds the same Qwen model in FP8 with ten 128K sessions and fits a small studio. It also holds a dense 27B model, Qwen3.8-27B or Qwen3.6-27B, in FP8 with 4 (7) such sessions, but at 1,344 GB/s its ceiling for one request at short context is about 47 tokens per second by our estimate, bandwidth divided by about 28 GB of weights read per token.

Mid: a team on one workstation. One RTX PRO 6000 Workstation Edition or Max-Q (96 GB) carries Qwen3.6-35B-A3B in FP8 for up to about eleven developers at the planned speed. For a developer who prefers the dense model, which scores higher in Qwen's tables, the same card holds Qwen3.8-27B in FP8 with 6 (13) sessions of 128K, with a ceiling of about 63 tokens per second for one request by our estimate (1,792 GB/s divided by 28.4 GB). Qwen3-Coder-Next fits with two sessions (five with an FP8 cache), a single-developer setup with the larger model. Two cards run Qwen3-Coder-Next as a TP=2 pair, as its card shows with --tensor-parallel-size 2, with about 20 planned sessions of 128K. Two Workstation Edition cards draw up to 600 W each and two Max-Q 300 W each; NVIDIA allows up to four Max-Q in one workstation, but three or four belong only in a chassis built for them with strong directed airflow, as our guide to a 2× RTX PRO 6000 Max-Q workstation explains.

Large: a department. Above about 20 developers a rack server with RTX PRO 6000 Server Edition cards takes over: two pairs on four cards give 36 planned sessions of 128K, four pairs on eight cards 72. The coding-agent server guide has the power, the H200 NVL option for Qwen3-Coder-480B and what to measure in a pilot.

We build RTX PRO workstations and AI servers for each of these tiers; send us the number of developers, the agents they use and the context they need, and our team replies with a configuration.

From one developer to a team server

On a shared server, vLLM replaces Ollama. Its stable documentation, dated 8 October 2026, lists an Anthropic Messages API (/v1/messages and /v1/messages/count_tokens) next to the Chat Completions and Responses APIs, so one engine serves all four clients. Start it with the tool parser from the model card, for example --enable-auto-tool-choice with --tool-call-parser qwen3_coder for the Qwen models and glm47 for GLM-4.7-Flash, as our guide to vLLM tool-calling parsers explains, and set --max-model-len to the context you sized for.

Prefix caching. An agent resends the same long beginning on every step, and vLLM reuses that part of the cache while it stays in memory. Claude Code prepends a short attribution block with its version and a fingerprint of the conversation to the system prompt; Anthropic's gateway reference says it has been stable within a conversation since version 2.1.181 and that CLAUDE_CODE_ATTRIBUTION_HEADER=0 omits it. By our reading, the shared system prompt can be reused across conversations only without that block, so test both settings in a pilot.

Keys and logging. vLLM's --api-key protects only paths under /v1, /v2, /inference and /cohere and manages no users. An LLM gateway with keys, budgets and a query log issues one key per developer; LiteLLM's proxy accepts requests in the Anthropic /v1/messages format and forwards them to the providers it supports. Claude Code's managed settings can pin the gateway: with allowedProviders set to ["customEndpoint"] and the gateway's ANTHROPIC_BASE_URL in the same file, Claude Code 2.1.285 or later refuses sessions pointed anywhere else.

Context limits. Claude Code cannot detect a lower limit that the server enforces, Anthropic's documentation notes. Distribute CLAUDE_CODE_MAX_CONTEXT_TOKENS with the managed settings so that every client compacts before the server rejects a request.

We set up vLLM and the gateway on the server as part of our Private AI/ML service; tell us how your developers work with agents today, and we propose the server and the model setup.

FAQ