On 26 October 2026 Cloudflare and the SGLang maintainers published a coordinated disclosure: from launch on 27 August until restoration on 9 September, one Workers AI model could, under a narrow condition, return fragments of another customer’s session. Cloudflare counted 155 affected responses. The model is patched. The buyer’s question is what every multi-tenant AI inference vendor must prove before you trust them with prompts that hold secrets. Checked against Cloudflare’s and LMSYS’s posts of 26 October 2026.
Disclosure: no vendor sponsored, reviewed or saw this piece before publication, and there are no affiliate links. Our independence policy explains how we work. How we review covers how we treat vendor claims.
What happened
Cloudflare’s incident report of 26 October 2026 (opens in a new tab) states that on 8 September 2026 researcher Gadi Evron of Knostic reported through HackerOne that very long prompts to GLM-5.3 on Workers AI (@cf/zai-org/glm-5.3) sometimes returned text from other users’ sessions. Cloudflare reproduced it the next morning, took the model offline within about forty minutes of declaring an incident, patched configuration and engine, and restored it the same evening. It says no other Workers AI model was affected.
The exposure window ran from launch on 27 August 2026 until restoration on 9 September. Cloudflare identified 155 customer responses that contained data from earlier sessions on the same model. Every one, it says, was to a prompt longer than 681,280 units of model input (the count vendors usually label “tokens”). After excluding its own tests and the researcher’s traffic, it reports timing and volume consistent with ordinary long-context use, and says it found no evidence of deliberate exploitation.
That is a cross-tenant leak in the sense that matters to buyers: Tenant A’s prompt or partial generation can surface inside Tenant B’s answer — not a public dump, but someone else’s confidential draft appearing in your reply as if the model had simply remembered it.
A small glossary
Multi-tenant inference means many customers share the same GPU serving fleet; isolation is a software promise, not a private machine each. A KV cache (key–value cache) is the model’s short-term working memory: sticky notes of intermediate “keys” and “values” so it need not re-read the whole prompt on every new word. Those notes live as GPU memory pages that engines often hand to the next request without wiping — overwrite on use — so ownership tracking has to be perfect. Prefix caching reuses already-computed KV for a shared opening; sharing it across tenants is a speed win and a confidentiality risk together. Cloudflare’s bug was a stale-page failure on a rejection path, not a marketed prefix feature, but buyers should ask about both. SGLang is the open-source inference engine Cloudflare used here in a disaggregated layout: prefill GPUs build the KV cache; decode GPUs generate the reply; the cache ships between them over a fast remote memory path.
Two faults, one leak
Two problems overlapped. First, misconfiguration: GLM-5.3 was advertised with a context window of 1,048,576 units, but the prefill KV pool held only 681,280. Cloudflare says it knew about the mismatch before launch and expected the engine to refuse anything it could not process; pre-launch tests returned successful HTTP responses, so the gap looked handled. Steal the lesson: HTTP 200 is not proof of coherent, isolated output. Measure pool size against context length from the running engine after every change to weights, quantisation or parallelism.
Second, an SGLang bug on the rejection path in disaggregated mode. When a prompt would not fit the pool, the scheduler marked it rejected and swapped in a one-unit placeholder so the batch could drain. On a single node that returned an error. In Cloudflare’s setup the admission queues treated a rejected request as still live: prefill ran the placeholder, transferred a scrap of KV, and signalled “transfer complete”; decode had reserved a full set of pages, received only the placeholder’s page, and attended over the rest — pages that still held earlier requests. The client got HTTP 200 and sometimes another tenant’s content. The LMSYS SGLang security update of 26 October 2026 (opens in a new tab) covers the same handoff failure and the upstream admission fixes.
What Cloudflare did well
Vendor accountability is not vendor-bashing. Credit where the sources earn it: a clear public report with a UTC timeline, impact numbers, and an honest admission that testing checked status codes rather than output integrity; fast containment the day of verification; coordinated disclosure with SGLang at RadixArk and thanks to the reporter; and upstream work for every other operator — admission fixes, deferred KV release, transfer bounds, optional checksums, and a recommendation to upgrade to SGLang v0.5.22 or later. Longer bets on page-level integrity tags and formal methods are direction, not a checkbox for today.
Buyer’s checklist: multi-tenant AI inference isolation
Ask for written answers, not slideware.
- KV pool versus context length. Do you measure the live KV pool against the advertised context window on every prefill and decode node after every weight, quantisation or parallelism change? What happens when they disagree — hard reject, silent truncate, or something else?
- Rejection and abort paths. If a request is rejected or a client disconnects mid-transfer, can decode still read pages that request never wrote? Show the engine version and the patches that retire rejected work before any KV handoff.
- Page wiping and ownership. Are KV pages zeroed between tenants, tagged per request, checksummed on transfer, or merely overwritten? Which of those are on by default for the models you sell us?
- Prefix and prompt-cache sharing. Is any prefix cache, prompt cache or speculative reuse shared across tenants? If yes, what namespace or cryptographic boundary stops Tenant A’s prefix from serving Tenant B?
- Disaggregation topology. Do you run prefill and decode on separate nodes with RDMA-style KV transfer? If so, which transfer engine, which deferred-release defaults, and which version pins them?
- Detection, not hope. Can you detect cross-request contamination in production (canaries, checksums, ownership tags)? What is logged when a check fails, and does the client get an error instead of a polluted 200?
- Tenant-scoped logging. Can you show a customer which of their responses may have been contaminated without reading other tenants’ prompts in the clear?
- Disclosure timeline. What is your committed window from confirmed cross-tenant leakage to customer notice and public technical detail? Who is the security contact, and is there a working bug-bounty path?
- Scope honesty. Will you state in writing which models, regions and serving modes share KV memory, and which are physically or logically isolated?
- Independent evidence. Can a customer or hired researcher run isolation tests in a staging tenancy with canary strings, under NDA if needed?
The Friday take
Shared GPU inference is how the industry makes long-context models affordable. Shared memory without ruthless ownership accounting is how a stranger’s brief appears in your completion like a ghost at the banquet. Cloudflare’s write-up is unusually complete; the 155-response count is finite and bounded to extreme lengths. None of that retires the bug class — it sharpens the shopping questions. If a vendor cannot answer the checklist without fog, treat their multi-tenant AI like a shared notebook whose erasable ink sometimes forgets to erase. Ask harder before you paste the crown jewels into the prompt box.
Sources checked 28 October 2026 (Europe/London). Primary posts: Cloudflare, 26 October 2026; LMSYS / SGLang, 26 October 2026.
Follow Cyber-Threat.net RSSFeedly (opens in a new tab)X (opens in a new tab)Bluesky (opens in a new tab)LinkedIn (opens in a new tab)