Kimi K2.7-Code on eight NVIDIA RTX PRO 6000 Blackwell Server Edition for coding agents

Kimi K2.7-Code on eight Server Edition cards: what fits

Kimi K2.7-Code runs on one server with eight NVIDIA RTX PRO 6000 Blackwell Server Edition cards: its 595.2 GB of INT4 weights fit the 664.3 GiB that eight cards keep for weights and cache by our users-per-card rule. What that server cannot hold is many long agent sessions. With decode context parallelism (DCP) over all eight cards it keeps about 12 sessions of 128K tokens in 16-bit, or about 25 with an FP8 cache, by our estimate; plain tensor parallelism holds one.

That makes the eight-card Server Edition node a fit for a team where 10 to 25 agents run at the same time, or for a pilot before a larger build. An engineering department of 40 to 60 developers who work with agents all day needs either eight H200 NVL or two Server Edition servers. Our guide to Kimi K2 hardware requirements sizes the K2.6 family for chat at 32K; this page sizes the coding model for the long sessions coding agents produce.

What Moonshot AI publishes for Kimi K2.7-Code

Moonshot AI created the Hugging Face repository on 11 June 2026. The card describes a model "With substantial improvements on real-world long-horizon coding tasks" and states that it "adopts the same native int4 quantization method as Kimi-K2-Thinking". The deployment of K2.5 and K2.6 "can be directly reused", in the card's words, because the architecture is the same.

moonshotai/Kimi-K2.7-Code model card, config.json and file list (sum of the 64 safetensors files, 595,177,988,208 bytes), read on 24 October 2026.

The config.json quantises the routed experts to 4-bit integers in the compressed-tensors "pack-quantized" format and keeps attention, the shared expert, the dense first layer, the output head and the vision tower in BF16. The card names vLLM, SGLang and KTransformers as recommended engines. vLLM's recipe for the model, updated on 29 July 2026, lists "8x H200 GPUs (verified)" or "equivalent aggregate VRAM (~640 GB)" and vLLM 0.19.1 as the minimum version. Eight Server Edition cards have 768 GB.

The card also states that "Kimi-K2.7-Code forces thinking and preserve_thinking as True". By our reading, the model's reasoning from earlier turns then stays in the conversation, so an agent session grows faster than its visible prompts and answers suggest. Images, such as a screenshot of a failing UI test, enter the same context as tokens.

Why long agent sessions decide the configuration

A coding agent calls the model again after every step, with the task, the files it read and the command output so far. Our server for coding agents explains the pattern and counts one generating session per developer at the busy hour as the upper bound. For K2.7-Code the cost of each session follows from config.json: 61 layers store 576 values per token (the 512-value latent plus the 64-value positional part), 70,272 bytes in 16-bit.

Our arithmetic from config.json (num_hidden_layers 61, kv_lora_rank 512, qk_rope_head_dim 64): 61 × 576 values × 2 bytes per token in 16-bit; FP8 as 576 bytes per layer, one byte per value, because vLLM 0.32.0 sizes a dense MLA cache as kv_lora_rank plus qk_rope_head_dim values of the cache data type; its 656-byte FP8 format with scales belongs to the sparse-MLA backends used for models such as DeepSeek-V3.2 and GLM-5.3.

vLLM's blog on decode context parallelism of 7 August 2026 states that under tensor parallelism "the latent KV cache is replicated in full across every TP rank". Each card therefore holds the whole cache of every session, and the 13.8 GiB per card that the weights leave on eight Server Edition cards is the budget for all sessions together. A full 256K session needs 17.16 GiB on every card and does not fit under plain TP=8.

Sessions on eight RTX PRO 6000 Server Edition cards

Our users-per-card rule takes 90 per cent of the 95.6 GiB the driver reports and subtracts 3 GiB per card: 83.0 GiB per card, 664.3 GiB for eight. The INT4 weights take 554.3 GiB, 69.3 GiB per card, which leaves 13.8 GiB per card. Three layouts change how much of each session lands on one card.

Our estimates by arithmetic, not measurements: concurrent sessions at full length, 13.8 GiB of cache per card, whole numbers rounded down. PP: pipeline parallelism, each card holds the cache of its own stage's layers. DCP: decode context parallelism, which splits each session's cache over the cards by token.

DCP is set in vLLM with --decode-context-parallel-size. The blog states that for MLA models the tensor-parallel size must be at least the DCP size and divisible by it, so eight cards allow DCP 8. Its benchmark ran Kimi K2.6 in NVFP4 on one node of eight B200 with an agentic trace whose median input is about 67K tokens: plain TP reached 100 per cent cache use at 64 concurrent requests and levelled off near 1,863 tokens per second per GPU, while DCP reached 6,091 at 512 concurrent requests with 82 per cent of the cache in use. The blog reports no time-to-first-token or per-token latency, and it names better support for MTP and speculative decoding as further work.

Treat the Server Edition figures as upper bounds. Our Kimi K2 guide records that vLLM's Kimi-K2-Thinking recipe left about 7.4 GiB per card less cache on eight H200 than our rule; the same shortfall on this card would leave about 6.35 GiB per card, about 5 sessions of 128K with DCP in 16-bit and 11 with an FP8 cache. Check the token count vLLM prints for the KV cache at start-up on the delivered server; our guide to vLLM KV cache size and maximum concurrency explains where to read it.

Kernels on Blackwell RTX (SM120) for this model

The RTX PRO 6000 has compute capability 12.0. vLLM's attention backend table, read on 24 October 2026, lists the MLA decode backends by compute capability. The dense-MLA backends FLASHINFER_MLA, CUTLASS_MLA and FLASHMLA are listed for 9.x or 10.x. Among the GPU backends for dense MLA, the one listed for "Any" capability is TRITON_MLA, with fp8 and fp8_e4m3 cache types and DCP support. On this card K2.7-Code would therefore run its attention through the Triton backend by our reading of the table, and we found no published speed for that path with a Kimi model.

The INT4 experts are a second item to test. Our comparison of vLLM, SGLang and TensorRT-LLM lists Marlin kernels built for 12.x for AWQ and GPTQ 4-bit weights in vLLM. K2.7-Code stores its experts in the compressed-tensors format instead, the recipe names no RTX PRO card, and no Moonshot AI document runs the model on one. Our page on Blackwell RTX SM120 software support tracks the engine releases.

We build eight-card Server Edition servers to order and run the sizing with your session lengths before we quote; send us the number of developers and the context your agents reach, and our team replies with the sessions per layout and a configuration.

Developers, sessions and the H200 NVL alternative

The developer count turns into sessions through the share of people whose agents generate at the same moment. A pilot's gateway log shows that share; until then we plan with one session per developer at the busy hour, as our coding-agent server guide does.

Our estimates by arithmetic, not measurements, upper bounds: sessions of 131,072 tokens; 13.8 GiB of cache per Server Edition card and 54.1 GiB per H200 NVL card (0.9 × 140.4 − 3, less 69.3 GiB of weights). Wall power: eight cards at 600 W, two processors and 15 per cent for the rest, as in our eight-card power guide; the same estimate for H200 NVL at 600 W.

vLLM's Kimi-K2-Thinking recipe measured 46.8 GiB of cache per card on eight H200, which gives about 43 sessions of 128K with DCP in 16-bit. Eight H200 NVL are therefore the configuration for 40 to 60 developers on one server; our guide to an eight-card H200 NVL server covers its two NVLink domains of four. Two Server Edition servers reach a similar count with an FP8 cache, as two replicas behind a load balancer, with twice the chassis and feeds. A team of 10 to 25 with agents at the same time fits one Server Edition server at the DCP figures, once the acceptance test confirms them.

Shorter sessions change the picture. At 64K, one Server Edition server with DCP 8 holds about 25 sessions in 16-bit. Many agents compact their context before it reaches the limit, and the compaction threshold of your agent framework sets the length to size for.

Serving Kimi K2.7-Code with vLLM

vLLM's recipe serves the model on eight H200 with --tensor-parallel-size 8, --tool-call-parser kimi_k2, --enable-auto-tool-choice and --reasoning-parser kimi_k2, and it states that the model "runs in thinking mode only with preserve_thinking forced on; keep the reasoning parser enabled". For agents that call tools, the parser decides whether tool calls reach the client as structured calls; our guide to vLLM tool calling parsers covers them.

The recipe's --mm-encoder-tp-mode data runs the vision encoder data-parallel, "to avoid TP communication overhead", in the recipe's words. By our reading each card then holds a full copy of the 400M-parameter encoder, about 0.75 GiB in BF16. For text-only agents the recipe says to pass --language-model-only "to skip loading MoonViT and free VRAM for KV cache". On eight Server Edition cards, where a 128K session costs about 1.07 GiB per card with DCP 8, that is close to one session.

Add --decode-context-parallel-size 8 for the DCP layout and set --max-model-len to the session length you plan. The recipe sets no cache type for the H200 path; an FP8 cache takes half the space of the 16-bit cache, and its effect on code quality belongs in your pilot. The 595 GB checkpoint plus the next version you want to test needs several terabytes of local NVMe. NVIDIA's configuration guide for NVIDIA-Certified Systems, updated on 30 September 2026, asks for system memory of at least twice the GPU memory, 1,536 GB for eight Server Edition cards. Our guide to RTX PRO 6000 Server Edition power at 600, 450 and 300 W has the feed table for an eight-card server, which needs a 400 V three-phase feed per side.

Modified MIT License and source code

Moonshot AI releases the code and the weights of K2.7-Code under a Modified MIT License. Its only modification applies when the software or a derivative is used in commercial products or services "that have more than 100 million monthly active users, or more than 20 million US dollars" (or the equivalent) in monthly revenue. Such products must "prominently display "Kimi K2.7 Code" on the user interface". The rest is the MIT text. A model served from downloaded weights on your own server sends no prompts or source code to Moonshot AI. Whether the clause applies to your company is for your legal department to assess.

What we supply

You can buy this system from us: our team sizes the configuration to your workload and budget and replies within one business day. We build AI servers to order with eight RTX PRO 6000 Server Edition cards or eight H200 NVL with their NVLink bridges, assembled and tested before delivery, on one EU contract and invoice with manufacturer warranty on every component. Quantities and lead time are confirmed in writing with the quote. Our page to buy an 8× RTX PRO 6000 Server Edition server lists the variants, and our overview of large LLMs on eight Server Edition cards places Kimi beside the other models. Deploying K2.7-Code with vLLM, the gateway and the agent integration is part of our Private AI/ML service, built by our engineering partner Vixen.UNO.

FAQ