2026-06-17 · Fuzzing · LLM serving

How GRIEF works

A greybox fuzzer that treats timed multi-request traces, not prompts, as the first-class input to LLM serving systems. The trace abstraction is where the bugs are; the staged confirmation pipeline is what makes the findings trustworthy.

Faultline team — Yunze Zhao · arXiv:2605.11202

TL;DR. Most fuzzers for ML systems mutate a single input, such as a prompt, a byte string, a token sequence, and ask whether the model output looks wrong. That abstraction can't reach the bugs we care about. For example, the KV-cache isolation failure behind CVE-2026-7141 requires two requests, the right scheduling overlap, and a freed block returning to the wrong tenant. No single prompt triggers it.

GRIEF mutates traces instead, traces are timestamped sequences of client events (SEND, CANCEL, DISCONNECT, WAIT) carrying prompts with stable family identities. It steers the search using runtime signals (latency, KV-cache events, scheduler pressure) and runs every candidate finding through a three-stage confirmation pipeline so scheduler jitter and decoding noise don't drown out real concurrency bugs. With this design, GRIEF has produced three CVEs against vLLM and SGLang to date, including a stale-state isolation failure, a single-client throughput-collapse pathology, and an availability-loss assertion in the LoRA scheduler.

GRIEF system architecture: a wrapper drives a core fuzzing loop (select seed, mutate, request shaping, evaluate, retain) against an inference engine (vLLM, SGLang) through adapters, with a separate confirmation path producing controlled replay and corpus growth. — GRIEF architecture: a serving-independent wrapper drives the core fuzzing loop; adapters translate abstract trace events into engine-specific calls. Reproduced from Zhao et al., 2026, Figure 2.

Why traces, not prompts

Serving-layer bugs depend on how requests overlap, not what any single request says. A KV block leaks because two requests landed in the wrong order on the recycle path. A latency cliff opens because one request's decode pattern deschedules the engine coroutine everyone else is waiting on. A scheduler assertion fires because four individually valid pressure conditions happened to co-occur for a single batch.

Mutating prompts can't reach any of these. The mutation surface has to be the schedule — when requests arrive, how they overlap, what state they share, when they cancel. That's what a trace captures.

The cost of this shift is that the input space is enormous and most mutations land somewhere benign. The whole rest of the design is about making that search tractable and the findings credible.

Trace structure

A trace is a timestamped sequence of client-side events. Each request event carries two identifiers:

request_id — the concrete transport instance, so a later CANCEL or DISCONNECT can target a specific in-flight request.
prompt_family_id — a semantic identity. Two requests in the same family are instantiated with identical prompt content across runs, adapters, and concurrency contexts. This is what lets divergence be attributed to serving behavior rather than prompt drift.

Prompts are constructed deterministically from the request shape and family ID: a shared structural prefix from the shape, a family-specific suffix from the family ID. Two requests with the same family get the same prompt; two with different families get different prompts that may still share a prefix. Either property can be turned into an oracle, consistency checks when prompts should match, contamination checks when they should differ.

Seed trace and result trace shown as ordered sequences of request and control events; below them the mutator panel shows timing mutation (reordering event offsets), event mutation (changing event types), and splicing (recombining segments of two parent traces). — A trace as fuzzing input. Three mutation classes operate on the same representation: timing, event, and splicing. Reproduced from Zhao et al., 2026, Figure 3.

Three classes of mutation

GRIEF explores the workload space through three trace-level operators:

Timing mutations

Preserve the event set, perturb event offsets. Changes whether requests co-batch, overlap during prefill, or re-enter the system during cache eviction. This is the operator that found the kv-cache overlap behind CVE-2026-7141.

Event mutations

Insert, delete, or modify lifecycle events — SEND, CANCEL, DISCONNECT, WAIT. Exercises cleanup, retry, and teardown paths that single-shot prompt fuzzing never touches.

Splicing

Recombines segments from multiple parent traces while preserving trace validity — rebasing timestamps, refreshing request IDs, removing orphaned control events. The directed variant aligns one trace's cache-warming phase with another trace's high-pressure window, biasing the search toward schedules that first populate shared serving state and then perturb it under load.

Catching real bugs without crying wolf

LLM serving is noisy. Scheduler jitter, queue placement, latency variance, and decoding ambiguity can all produce one-off anomalies even when the implementation is correct. A naïve fuzzer would file every divergence as a bug and drown the maintainer.

GRIEF uses a three-stage confirmation pipeline that separates low-cost suspicion generation from high-confidence reporting.

Stage 1 — Behavioral checks

Runs after every trace execution. Treats the server as a black box and asks: is the observed request/response behavior consistent with the API contract and with the lifecycle the trace encodes? Hits include request timeouts, scheduler stalls, severe TTFT regression, lifecycle violations, corrupted outputs, and unrecovered KV usage. A behavioral hit doesn't produce a finding, it produces a candidate.

Stage 2 — Logprob-assisted relational confirmation

The candidate trace is replayed with deterministic decoding and log-probability reporting enabled. If the replay matches the original execution, the candidate is dismissed. Otherwise, GRIEF inspects the first divergent token and asks whether the original token is still a near-tied candidate under the replay distribution. Near-ties are benign numerical ambiguity. A large probability gap means the original execution was on a different path, i.e. stale state, cross-request contamination, or scheduler-induced divergence.

# Algorithm 1 — paraphrased from the paper.
# Inputs:  y  = original tokens
#          y' = replay tokens
#          L  = replay logprob distributions per position
#          N  = top-N candidate count
#          ε  = near-tie tolerance
p = first_difference(y, y')
if p is None:
    return PASS                      # replay matches; nothing to confirm

top_N = topN(L[p], N)                # top tokens at the first divergent position
delta = L[p][y'[p]] - L[p][y[p]]     # replay's advantage over the original

if y[p] in top_N and delta < ε:
    return FALSE_POSITIVE            # near-tie; benign decoding ambiguity

return TRUE_POSITIVE                 # original token now unlikely → real divergence

The rule is deliberately conservative. It avoids reporting cases where nearly tied logits could legitimately decode differently while preserving sensitivity to the exact failure modes we want to catch: stale KV state, cross-request contamination, and scheduler-induced execution paths that make the original token implausible under clean replay.

Stage 3 — Structural KV forensics

For high-confidence attribution, GRIEF observes the server's KV-cache block lifecycle through an out-of-band event stream. This stage detects structural anomalies, such as cross-adapter block reuse, hash-content conflicts, cross-run block-snapshot divergence, and including cases where the corruption has not yet become visible in output. A structural finding is filed only when the anomaly reproduces in at least ⌈2k/3⌉ of k re-runs. The summary doubles as a compact fingerprint for deduplication and offline diagnosis.

What came out

Three bugs have come out of GRIEF so far, each in a distinct impact class. They are all on the CVE page with disclosure links.

CVE-2026-7141 — state corruption / isolation failure

Found by a timing mutation that overlapped a victim with an attacker on the kv-block recycle path. The structural stage caught it before the output corruption was even consistent. Full writeup: A KV-cache that forgets to forget.

CVE-2026-9540 — performance pathology / cross-request interference

A single attacker client, one request in flight at a time, using documented API parameters. Victim TTFT went from a 13 ms baseline to a p50 of 18 s (a 1361× delay), and ~56,955 victim requests were never served during the 47-minute attack window. The server never crashed, never logged an error. Behavioral stage flagged the TTFT regression; logprob confirmation isn't needed for a latency bug, but the trace replay was.

CVE-2026-10300 — availability loss / scheduler invariant violation

GRIEF reached the SGLang LoRA scheduler crash in ~2 minutes of fuzzing on a single H100, after fewer than 200 iterations. The crashing trace combines four valid pressure conditions — high KV-cache occupancy, mixed prompt and prefix lengths near chunked-prefill boundaries, simultaneous BASE/LoRA admission, and a burst of closely spaced LoRA arrivals. Each condition is accepted in isolation. The crash only fires when they co-occur in the same transient scheduling state. Directed splicing got us there.

The pattern is the same across all three: none of them are reachable from a single prompt. Each requires a specific shape of concurrent request schedule, and each is silent until the right overlap happens. That is the surface trace-level fuzzing was built to cover.

What this is, and what it isn't

GRIEF is an initial framework for serving-layer fuzzing, not a complete one. The trace abstraction is general; the adapters, oracles, and mutation operators are calibrated to a specific corner of the serving stack, single-GPU vLLM and SGLang with the optimizations we've modeled (prefix sharing, LoRA, speculative decoding, MoE). Multi-node deployments, autoscaling, and KV-cache offloading aren't covered yet. The oracle pipeline is tuned to favor false negatives over false positives, which means GRIEF probably misses real bugs whose first divergent token has a near-tied distribution under replay. We think these are the right tradeoffs for a tool whose findings have to be actionable enough to file as CVEs, but they are tradeoffs.

The broader bet behind the design is that concurrent serving behavior is a first-class security surface for LLM infrastructure — distinct from model safety, distinct from API conformance, and not reducible to either. Trace-level fuzzing is one way to test that surface. We expect there are others.

References

Paper — Continuous Discovery of Vulnerabilities in LLM Serving Systems with Fuzzing (arXiv:2605.11202). §3 covers the fuzzing system design described here; §4 has the full evaluation.
A KV-cache that forgets to forget — companion post on CVE-2026-7141, the bug GRIEF's timing mutator found.
CVE index — all three GRIEF-discovered CVEs with disclosure and patch links.