Prefill-Decode Disaggregation in SGLang Leaked KV Cache Across Requests. No Scanner Will Tell You to Upgrade.

Prefill-decode disaggregation splits LLM serving in two: one pool of GPUs reads the prompt and builds its KV cache, then ships that cache over RDMA to a second pool that generates the answer. On October 26 Cloudflare and the SGLang maintainers published a coordinated disclosure showing what happens when the two halves disagree about which cache pages belong to whom. Cloudflare's GLM-5.3 model on Workers AI returned fragments of other customers' requests in 155 responses between August 27 and September 9.

The incident is Cloudflare's. The exposure is wider, and the part that matters for anyone self-hosting is what the announcements do not say: there is no advisory to make your tooling notice it.

What shipped

The engine fixes are spread across three SGLang releases. Comparing tags against merge commits in the repository puts the admission fix that stops the Cloudflare failure (pull request 36988) in v0.5.20, and the earlier admission rejection (38935), the optional transfer checksum (39500) and the default-on deferred KV release (41023) in v0.5.21. PyPI shows v0.5.20 uploaded on September 18, v0.5.21 on October 1, and v0.5.22, which adds MORI backend coverage and the release metrics, on October 16.

SGLang's own post goes further than Cloudflare's. It says the related abort race, where an aborted request's in-flight RDMA writes land in pages already handed to another request, affected default disaggregated configurations from the v0.4.x line until v0.5.21, or v0.5.22 on MORI. It also says the fix line was not backported to earlier release branches, and that an operator has no reliable way to tell whether exposure already happened, because the corruption raises no error and writes no log line.

What it changes if you already run it

Cloudflare's report explains why this was reachable at all. SGLang sizes the KV pool at startup from whatever GPU memory is left after loading weights, and the advertised context length is a separate setting. Nothing forces them to agree. Cloudflare's prefill pool held 681,280 tokens against a 1,048,576-token context, and its pre-launch tests checked HTTP status codes and throughput, not whether the output made sense. Pages are not zeroed between requests, so a decode node reading pages its request never wrote reads someone else's data.

That testing gap is the one we describe in From PoC to Production: an evaluation that cannot catch a wrong answer is not an evaluation. A 200 with coherent text was the failure, not the success signal.

The source answers two questions the posts leave to the reader. In v0.5.22, disaggregation_enable_kv_checksum is declared with a default of false, and its help text says it must be enabled on both prefill and decode. Zeroing pages on free exists only as draft pull request 46435, unmerged. So an upgrade makes the known paths safe, but the detector for the next bug of this shape stays off unless you turn it on.

What it breaks: your scanner's silence

This is the claim the disclosures do not make. The GitHub Advisory Database lists ten reviewed advisories for sglang on pip, and the highest affected-range ceiling among them is 0.5.12. None covers this exposure. An OSV query for sglang 0.5.19, the last release before the incident fix landed, returns no vulnerabilities at all. Any scanner that reads those databases has nothing to flag, so a lockfile pinned to 0.5.19 passes audit today.

The disclosure also has a quieter history. Issue 30233, filed by an outside user on July 6, described the exact mechanism, an over-length prompt swapped for a one-token placeholder and decode reading the rest of the reserved pages, and included a proposed patch. It was filed and handled as garbage output, and closed on September 5, three days before the researcher's report reached Cloudflare. A correctness bug in shared memory is a data-isolation bug whether or not anyone labels it one.

The pins are out there. In a sample of 40 code-search hits pairing the disaggregation launch flag with the official image, third-party deployment guides from dstack and OME pinned v0.5.10.post1 and v0.5.5.post3. Others used model-named tags such as kimi-k3, which carry no version number to check against the fix list.

The falsifiable version: if a GitHub or OSV advisory covering sglang below 0.5.21 is published, scanners will start catching old pins and this section is wrong. Until then, upgrading is a manual decision.

Verdict: upgrade now, then check two numbers

If you run SGLang with --disaggregation-mode, upgrade to v0.5.22 or later. SGLang recommends that, and Cloudflare recommends the same floor. Then:

1. Read the version inside the image, not the tag:

docker run --rm --entrypoint pip lmsysorg/sglang:YOUR_TAG show sglang

1. On every prefill and decode node, compare the KV pool to the context length. The v0.5.22 scheduler prints both on one startup line; the last column is 1 when the pool fits and 0 when it does not:

grep -o 'max_total_num_tokens=[0-9]*.*context_len=[0-9]*' prefill.log | awk -F'[=,]' '{print $2, $NF, ($2+0 >= $NF+0)}'

Any 0 means lower the context length or free memory. Re-run it after every change to weights, quantization, parallelism or memory fraction.

1. Pass --disaggregation-enable-kv-checksum to both roles if your model is covered; Cloudflare says coverage does not yet extend to GLM 5.3, GLM 5.3 Flash or DeepSeek V4, so confirm in the logs that requests are actually being verified. Watch sglang:decode_deferred_kv_release_total by outcome. SGLang says a healthy deployment releases almost all holds on drain confirmation rather than timeout.

2. If you cannot upgrade yet and run v0.5.19 or v0.5.20, SGLang's interim advice is to set SGLANG_DISAGGREGATION_DEFERRED_DECODE_KV_RELEASE=1 on both roles, and it is explicit that this leaves non-decode-initiated failures unprotected.

3. Add a hand-written check to your dependency policy: sglang below 0.5.21 in any disaggregated deployment fails review, whatever the scanner says.

Single-node deployments were not on the path either post describes. If you never split prefill from decode, this release changes nothing for you.