A KV-cache that forgets to forget

TL;DR. vLLM's V1 engine has a KV-block zeroing pipeline that's supposed to wipe freed cache blocks before they are recycled. The wipe is gated on has_mamba_layers() — it only runs for Mamba models. For every FullAttention model in the wild (Llama, Qwen, Mistral, …), the gate evaluates false and the wipe is skipped. The next request to receive those blocks decodes from stale K/V left by a previous tenant. Output is fluent, confident, and wrong.

We caught this with GRIEF, our greybox fuzzer that treats timed multi-request traces as first-class inputs to LLM serving systems. This post focuses on the bug itself; the companion post walks through how the fuzzer works.

The bug

When a request completes (or a client cancels mid-decode), its KV blocks return to the free pool. They are immediately eligible for the next request. PR #35219 introduced a zeroing path to prevent stale data from leaking forward. The path lives in vllm/v1/kv_cache_interface.py. But it's gated:

# vllm/v1/kv_cache_interface.py — paraphrased
def needs_zero_init(self) -> bool:
    # Originally added for Mamba's recurrent state — but
    # FullAttention's KV blocks need wiping too.
    return self.has_mamba_layers()

# Block-recycle hot path
if cache_config.needs_zero_init():
    zero_blocks(freed)        # only Mamba ever hits this
free_pool.extend(freed)       # blocks return dirty otherwise

For Mamba models the gate fires, the wipe runs, and isolation holds. For every other architecture, the gate returns false, the wipe is skipped, and freshly-allocated blocks contain whatever K and V the previous tenant left behind. The next decode step reads that as if it were valid attention context.

Evidence

Three findings from the disclosure issue make the case concrete:

To rule out prompt injection, we replayed each affected schedule with three different attacker prompts: the original fuzzer-generated string, a repeated synthetic fingerprint, and a long sequence of digit tokens. For each victim slot, the corrupted output was bit-identical across all three attacker prompts. The corruption is determined by the victim's prompt and its schedule position, not by what the attacker said. That's state contamination, not content leakage.

Status & fix

The proposed fix removes the has_mamba_layers gate so the zeroing path runs for every architecture (PR #39283). As of publication, the PR is open and not yet merged.

• CVE: CVE-2026-7141
• CWE: CWE-908 (Use of Uninitialized Resource)
• CVSS v3.1: 5.1 medium · v4.0: 2.9 low
• Affected: vLLM ≤ 0.19.0, all FullAttention model families

Why this matters

KV-cache reuse is the entire reason inference engines hit modern throughput numbers. Block recycling is a hot path that runs on every completion and cancellation. A missing zero in that path silently breaks tenant isolation across every co-scheduled request, with no crash, no error log, no malformed response — just a confidently wrong answer with the right format.

Production guardrails won't catch this. Rate limits don't apply. Output filters see well-formed text. Logprob-based anomaly detection sees high-confidence tokens. The only effective signal is the one GRIEF used: replay the same trace and watch for divergence that shouldn't exist.

We think the broader lesson is that concurrent serving behavior is a first-class security boundary for LLM infrastructure, distinct from model safety and from API conformance. The paper has more on the threat model and on two additional bug classes — performance pathology and scheduler-state drift — that fall out of the same testing approach.