DSpark Speculative Decoding: The 4x Was Measured on Math, and Tool Calls Accept About Half as Many Draft Tokens

Speculative decoding sells one number: how many times faster the model writes. On 15 September the vLLM team published a DSpark drafter for Kimi K3, trained with its Speculators library, that takes single-stream decode on math reasoning from about 110 to about 435 tokens per second per user, and up to roughly 3.5 times the output throughput under concurrent load. The same post names drafters for Qwen3.6-35B-A3B, Gemma-4-31B and GLM-5.2. The cards for the first two deploy them at tensor parallelism 2 and 4, which puts them within reach of a self-hosted team.

The headline is accurate. It is also the best case, and the model cards say so in their own tables.

The claim, stated fairly

DSpark comes from a DeepSeek-AI and Peking University paper posted on 6 July. It keeps the one-pass parallel drafting of DFlash, adds a small sequential head so drafted tokens can condition on each other, and adds a confidence head that estimates which drafted tokens will survive, so a scheduler can verify long prefixes when the GPU is idle and trim them when it is busy.

The Kimi K3 drafter proposes eight tokens per step. The Hub counts 4.74 billion parameters in it. The vLLM post reports a macro-average acceptance length of 4.11 across nine evaluation domains, 6.42 on math reasoning, and aggregate throughput rising from 177 to 683 tokens per second as concurrency goes from 1 to 16, a figure the post does not tie to a domain. The single-stream figure in the headline, and the throughput chart, are both labelled math reasoning.

What the acceptance tables cover

Acceptance length here counts the target model's own token as well, so a drafter that never lands a guess scores 1.0. The Qwen card's per-position rates sum to one less than its average, to rounding, which confirms it. The gain is therefore the distance above 1, not the number itself.

Each of the four drafter cards publishes the same nine subsets. On Kimi K3, math reasoning accepts 5.42 extra tokens per verification step and tool calling accepts 2.33, which is 43 percent of the math figure. Writing, at 3.14, and general questions, at 3.21, are lower still. The pattern repeats on every card: tool calling earns 59 percent of math's gain on Qwen3.6-35B-A3B (3.39 against 5.03), 52 percent on Gemma-4-31B (3.21 against 5.24), and 56 percent on GLM-5.2 (3.39 against 5.27). Retrieval-augmented prompts sit between 3.44 and 4.01.

The independent source agrees on the direction. The DSpark paper reports Qwen3-4B accepting 5.57 on math, 5.12 on code and 3.49 on open-ended chat, and trained its drafters on a mix that is 39.4 percent math and 38.9 percent code. It evaluated three domains: math, code and chat. Tool calling was not one of them.

What the speed charts leave out

Acceptance length is not speed. It is the input to speed, and every speed chart published for these drafters is a math chart. The Kimi chart is labelled math reasoning. The Gemma card carries one latency chart, also math reasoning. The Qwen card carries none.

The GLM-5.2 card also carries one chart, and it is the same uploaded image file as the Gemma card's, titled google/gemma-4-31B-it. As of today the GLM-5.2 drafter has no speed measurement of its own, only a Gemma chart attached to its page.

The evaluation set behind the tables is small and uneven. In the RedHatAI/speculator_benchmarks dataset, math reasoning is 80 prompts and tool calling is 200. The tool-calling file was added on 15 April and is not among the eight files the dataset card describes, so its provenance is undocumented on the page that hosts it.

Where the speedup flips

Domain is one boundary. Concurrency is the other, and vLLM's own pull request measured it.

The first DSpark support in vLLM, merged on 1 July, listed confidence-based scheduling as out of scope. It arrived on 12 August in pull request 47808, which reports that on DeepSeek-V4-Flash at 256 concurrent requests, verifying a fixed seven draft tokens ran 33 percent below no speculation at all. The adaptive mode stayed within 3 percent of fixed-length drafting up to 64 concurrent requests and kept the win above it. The author flags that the no-speculation run shared its node with another job, so that baseline is somewhat depressed, which makes the 33 percent loss conservative rather than inflated.

That fix is a flag, enable_adaptive_verification, and its default is False in the v0.29.0 release and on main. None of the four drafter cards' deploy commands sets it, and neither does the Kimi K3 command in the vLLM post. Copy the published command, put agent traffic through it at high concurrency, and you are running the configuration that, on DeepSeek-V4-Flash, the pull request measured losing to no speculation.

What survives production

The paper names its own limit: every request pays the drafter's fixed cost for a full block, and for low-acceptance prompts that cost is not recovered. Tool calls and retrieval, the prompts an agent loop is built from, are the low-acceptance rows in these tables, and agents send them concurrently.

So the central claim of this post, falsifiable: on tool-calling traffic, a DSpark drafter from this family delivers roughly half or less of the math headline's per-user speedup, and at high concurrency with the default configuration it can be slower than serving without it. A tool-call latency chart from the Speculators team showing a speedup near the math figure would prove that wrong.

This is the recurring shape of why enterprise AI does not ship: the benchmark was real, it was measured on the workload that flatters the method, and the gap only shows up when your own traffic arrives. Sizing that gap before buying GPUs for it is most of what a private LLM deployment is.

What to run before you adopt it

1. Serve your target with the drafter, then run the Speculators evaluator against it on the subsets that look like your traffic, with math included as the control:

   python scripts/evaluate/evaluate.py --target http://localhost:8000/v1 sweep --subsets tool_call,rag,math_reasoning
   

2. Set the enable_adaptive_verification key to true inside the --speculative-config JSON, on vLLM 0.28.0 or later where the flag first shipped, and run the sweep again.

3. Keep the drafter only if median inter-token latency improves at your peak concurrency, not at a single stream. If the gain on your tool-call subset is under half the math subset's, budget capacity from the tool-call number.