Cloudflare GLM-5.3 Workers AI model exposed customer data across requests

What happened

On September 8, 2026, security researcher Gadi Evron from Knostic reported a tenant isolation issue affecting Cloudflare's GLM-5.3 Workers AI model through the HackerOne program. Between August 27 and September 9, a small number of customers using the GLM-5.3 model (@cf/zai-org/glm-5.3) received responses containing data from other customers' requests made to the same model shortly before theirs. In total, 155 responses were affected, all to prompts longer than 681,280 tokens.

Cloudflare reproduced the issue on September 9 and took the model offline within 40 minutes of declaring an incident. After deploying a corrected configuration and patched inference engine, the company restored service the same day. No other model hosted on Workers AI was affected.

Analysis of the affected requests showed activity consistent with ordinary long-context use rather than deliberate exploitation. After excluding Cloudflare's testing and the researcher's verification, no evidence of intentional abuse was found.

Root cause: misconfiguration and upstream bug

The data exposure resulted from two overlapping issues. The first was a configuration mismatch in SGLang, the open-source inference engine Cloudflare uses. GLM-5.3 was deployed with an advertised context window of 1,048,576 tokens, but the KV cache pool on prefill GPUs only had capacity for 681,280 tokens. Cloudflare was aware of this mismatch before launch but believed the engine would gracefully handle the discrepancy. Pre-launch testing showed successful HTTP responses and no error frames during benchmarking, leading the team to assume the mismatch was not problematic.

The second issue was a bug in how SGLang handled prompts longer than the KV cache pool in disaggregated prefill/decode mode. When a prompt arrived that was longer than the pool could hold, SGLang's scheduler rejected it and replaced it with a one-token placeholder. However, in disaggregated mode, the rejection path did not properly abort the request or return an error to the client. Instead, prefill generated KV cache for the placeholder prompt and transferred it to the decode node, which assumed the full prompt had been processed.

The decode node subsequently generated responses while attending over stale cache pages containing data from earlier requests. This meant responses to long prompts returned text from other users' sessions, arriving faster than expected and with seemingly coherent content that actually came from previous requests' KV cache pages.

Immediate remediation steps

To fix the configuration issue, Cloudflare adjusted the GLM-5.3 configuration to match GLM-5.2, which freed enough GPU memory so the prefill KV cache pool now holds approximately 2.04 million tokens—roughly double the advertised 1,048,576-token context length. This ensures valid requests never reach the rejection path.

To fix the engine, Cloudflare also made the rejection path safe by patching SGLang so that requests carrying pending rejections are retired at the disaggregation admission queue with their original error before any bootstrap, allocation, or KV transfer occurs. This change was proposed to upstream as PR #38704 and included decode node verification that the number of transferred tokens matches reserved tokens.

Cloudflare also deployed broader hardening across all Workers AI models. Every disaggregated deployment now requires KV pool size to be larger than context length on both prefill and decode, and checks are performed directly from the running engine before any deployment change ships. These patterns are applied to every model served with SGLang.

Impact on other Workers AI models

GLM-5.3 was the only model affected by this bug. Every other disaggregated Workers AI deployment has KV pools larger than its context length on both prefill and decode nodes, meaning no request could reach the rejection path that triggered the vulnerability.

Cloudflare has applied both the engine patch and hardening checks to every model served with SGLang. The fixes are now required for all new deployments.

Upstream fixes and recommendations

The bug had been reported to SGLang by another user as issue #30233 on July 6, 2026, which Cloudflare discovered during its investigation. Cloudflare had already been flagging similar issues with SGLang since v0.5.10, and this incident accelerated collaborative work with the SGLang maintainers.

The patch for this issue was released in SGLang v0.5.20, with follow-up improvements in v0.5.21 and v0.5.22. Cloudflare strongly recommends all operators using SGLang update their dependencies to v0.5.22 or later. The fixes are available for all inference engine operators dependent on SGLang to deploy.

Disclosure and investigation details

This disclosure is coordinated with SGLang's project maintainers and published alongside their technical post describing the bug and collaborative hardening efforts. The timeline of events shows the incident was declared at 13:22 UTC on September 9, 2026, with reproduction occurring at 08:00 UTC that same day, just over 8 hours after the HackerOne report was received.

Cloudflare validated both configuration and engine changes against the researcher's reproduction script and internal canary-marker tests before restoring the model to production.

Source

AI-assisted, verified against the source. This post was drafted with AI from the primary source linked above and checked automatically against it. For anything you depend on, read the original.