vLLM tool calling: parsers per model family, tool_choice and strict mode in vLLM 0.32

What vLLM needs to return tool calls

A model emits a tool call as text in its own format: JSON inside tags for Hermes-style models, Python-like calls for Llama 4, the Harmony format for gpt-oss. vLLM's OpenAI-compatible server converts that text into the tool_calls field of the response only if you start it with --enable-auto-tool-choice and the --tool-call-parser that matches the model family. Without the right parser, the call arrives as plain text in content, and the agent framework sees no tool call at all.

vLLM enforces calls that parse only with a named tool, tool_choice: "required" or strict mode; in the default auto mode the model's text is parsed as it comes.

Everything below comes from vLLM's tool-calling documentation for release 0.32.0 (page dated 8 October 2026) and the development documentation of 23 October 2026, read on 24 October 2026, unless another source is named. vLLM releases every two weeks or so, and parser names and defaults change with them, so check the page for the release you pin.

Three server flags matter. --enable-auto-tool-choice lets the model decide on its own when to call a tool; --tool-call-parser names the parser; --chat-template is optional and points to a template that "handles tool-role messages and assistant-role messages" with tool calls, for checkpoints whose own template does not. The documentation's quick start serves meta-llama/Llama-3.1-8B-Instruct with --tool-call-parser llama3_json and the template tool_chat_template_llama3.1_json.jinja from vLLM's examples. A fourth flag, --tool-parser-plugin, registers a parser you write yourself.

Which parser for which model family

The parser follows the model family, not the model size. vLLM's documentation lists about thirty parsers; this table covers the families companies most often run on our servers, with the caveats the documentation gives.

vLLM tool calling documentation, release 0.32.0 (8 October 2026) and development version (23 October 2026), read 24 October 2026. Qwen3 from Qwen's function-calling guide; DeepSeek-V4-Flash, GLM-5.3 and Gemma 4 from vLLM's recipes and the Qwen3-Coder-Next card's qwen3_coder, as cited in our guide to choosing an open LLM for a company assistant and our coding-agents server page.

Three entries need a word. For gpt-oss, the openai parser reads the Harmony format the models are trained on. For Qwen3-Coder, the documentation lists qwen3_xml, while the model cards, including Qwen3-Coder-Next, start vLLM with qwen3_coder; the 0.32.0 release notes make qwen3_xml an alias of qwen3_coder (pull request 60136), so from that release both names select the same parser. For Llama 3.1 and 3.2, the documentation warns that "Parallel tool calls are not supported for Llama 3" and that the model may serialise arrays as strings, so an argument typed as a list can arrive as "[1, 2]". It also warns that "Llama's smaller models frequently fail to emit tool calls in the correct format" with the pythonic parser.

Release 0.32.0 added an hf parser that reads the response template shipped in the checkpoint's tokenizer_config.json. The development documentation adds that it "cannot be combined with other reasoning or tool call parsers" and rejects strict tools, required or named tool choice and parallel_tool_calls=false, so it suits a quick test more than an agent in production.

We build GPU servers and deploy open models on them with vLLM; send us the model family you plan and the tools your agents will call, and our team replies with a configuration and the parser and template settings to start from.

tool_choice: auto, required, named and none

The tool_choice field of the request decides whether the model may, must or must not call a tool, and with it whether vLLM constrains the output.

vLLM tool calling documentation, release 0.32.0, read 24 October 2026; required from vLLM 0.8.3.

With a named function, "vLLM will use structured outputs to ensure the response matches the tool parameter object", and the documentation warns of "several seconds of latency (or more)" on first use while the grammar compiles. With required, the model must produce one or more tool calls, how many depending on the query. Both constrain syntax only: the documentation is explicit that you get "a validly-parsable function call - not a high-quality one". The model can still choose the wrong function or invent an argument value.

auto is what agent frameworks send, and it is the weak spot. In release 0.32.0, "the model generates tool calls without any grammar and malformed markup can leak into the response" unless strict settings apply. With none, tool definitions still go into the prompt by default; --exclude-tools-when-tool-choice-none removes them, which saves prompt tokens on calls that do not need tools.

Strict mode: strict: true and --tool-strict-level

Strict mode brings constrained decoding to auto. There are two switches, one per tool in the request and one for the whole server.

Per tool. A tool definition with strict: true pins that tool's argument schema; the documentation stresses that "a tool that omits strict keeps its arguments unconstrained", even when another tool in the same request is strict. It recommends OpenAI's strict-schema style: additionalProperties: false, every field listed as required, optional fields typed as nullable. The same page notes that "Most OpenAI-compatible clients and agent frameworks never set strict on their tools", so check what yours sends before relying on it.

Per server. --tool-strict-level takes three values. auto, the server default in 0.32.0, follows the request's tool choice and each tool's strict flag. function will "Constrain the tool-call envelope (markup and the function name) for every request with tools". parameter will "Additionally pin argument schemas for every tool, as if every tool had strict: true." The environment variable VLLM_ENFORCE_STRICT_TOOL_CALLING defaults to true; set to false, it switches these constraints off regardless of strict.

The development documentation of 23 October 2026 adds parser defaults: when the option is unset, "the tool parser's default level applies: function for glm47, auto for the other parsers", and passing --tool-strict-level auto overrides it. The 0.32.0 page names no such parser default, so expect this from a later release and check the documentation of the version you run.

Which level to choose depends on the tools. function stops malformed markup and invented function names at little cost to the model's freedom. parameter also forces every argument into its schema, which suits internal APIs with fixed schemas. Test both on a sample of real agent tasks and count wrong tool choices, wrong arguments and calls returned as text.

Reasoning models and tool calls

Models that think before answering, such as Qwen3 in thinking mode or DeepSeek-R1, need a reasoning parser next to the tool parser, set with --reasoning-parser. vLLM's reasoning documentation for 0.32.0 (dated 1 October 2026) lists names such as deepseek_r1, qwen3, glm45, gemma4 and step3p5, and states that "tool calling only parses functions from the content field, not from the reasoning". A call the model writes while still thinking is therefore not executed. The development documentation gives a combined example for MiMo models: --tool-call-parser mimo, --reasoning-parser mimo and --enable-auto-tool-choice with the checkpoint's own chat template.

Constrained decoding and reasoning interact. The reasoning page says the structured-output engine uses the end-of-reasoning token to check whether reasoning is present, and the structured-outputs page warns that with Qwen3 Coder and reasoning enabled, "structured outputs might become disabled if the reasoning content does not get parsed"; its fix is the enable_in_reasoning option of --structured-outputs-config. If strict calls fail only when thinking is on, check the reasoning parser first.

Structured outputs when no tool is involved

Many "tool" uses are extraction tasks: a ticket in, a JSON object with category and priority out. That needs no tool parser. vLLM's structured outputs, with the xgrammar or guidance backends chosen automatically per request, constrain a reply to a JSON schema through response_format with json_schema, or through the structured_outputs field with json, regex, choice, grammar or structural_tag. The older guided_json fields were removed in vLLM 0.12.0. For a fixed output format this is simpler and more reliable than a single-tool auto call, and it works with any model, also one without a tool parser.

When tool calls arrive as text

Five causes follow from the documentation, and each can be checked on the server.

1. No auto tool choice. Without --enable-auto-tool-choice, auto requests cannot return tool calls. Check the command line or the container definition the server was started with.
2. Wrong parser. A hermes parser on a model that writes Python-style calls finds nothing to parse, and the call stays in content. Match the parser to the family in the table above, or to the model card; release 0.32.0 also checks tool parser compatibility at start-up (pull request 59749), so read the start-up log.
3. Wrong or missing chat template. Llama 3.1 and several other families need the template named in vLLM's documentation, which formats tool-role and assistant-role messages for the model. Mistral checkpoints served through the Transformers backend need the provided template because the default "requires 9-digit tool call IDs".
4. Reasoning not separated. A model in thinking mode without --reasoning-parser mixes reasoning and calls in content, and the tool parser may fail on the mix.
5. Unconstrained auto. Malformed arguments in auto mode are expected behaviour without strict settings. Set strict: true on the tools or raise --tool-strict-level.

Tool calling also costs capacity. Every agent step resends the tool definitions with the conversation, so a long tool list adds prompt tokens to every call; our guides to AI agent GPU sizing and to servers for LLM agents with long context turn that into GPU memory and prefill time. For a workflow tool connected to vLLM, our article on AI agents with n8n and a private LLM covers the client side, and our article on MCP security the tool servers behind it.

Parsers, versions and security

A tool parser reads text the model produced, and a model can be steered by its input. Advisory GHSA-79j6-g2m3-jgfw (CVE-2025-9141, rated High, 8.8, published 20 August 2025) described remote code execution in the qwen3_coder parser: for parameters of undefined or unrecognised type, the parser called Python's eval() on the argument, so an authenticated user who could make the model pass code as an argument could run it on the server. It affected vLLM from 0.10.0 up to, but not including, 0.10.1.1, which fixed it, and only servers started with --enable-auto-tool-choice and that parser. Our page on vLLM security advisories and which version to run lists the advisories of 2026 by the way an endpoint is exposed.

The tool calls themselves are a second risk. vLLM only formats the call; your application executes it, with whatever permissions it holds. Give agents read-only tools where possible, validate arguments again before execution and ask a person to approve calls that change data, as our article on prompt injection explains. Our Private AI/ML service, with engineering by our partner Vixen.UNO, deploys open models with vLLM and connects them to company systems; describe the workflow your agents should handle, and we propose the model, parser settings and server.

FAQ