Loading
Life sciences · Preprint
arXiv · September 4, 2026
Early or partial results. Treat as a signal, not a conclusion.
This measurement study reports that enabling prefix caching in LLM serving systems increases run-to-run divergence sharply as a function of weight quantization, from 36.2% of episodes affected at 16-bit to 75.0% at 4-bit precision. The authors trace the divergence to unmanaged cache state and show that cached serving is deterministic when cache state is controlled, but irreproducible in practice because that state is not reset by default. The finding is reproducible across two serving engines but remains a single-site systems measurement without comparison to alternative cache management strategies or a causal intervention study.
Observational empirical systems measurement with controlled parameters. Multi-turn agentic tool-use workload run against LLM serving systems; no human subjects or clinical population.. Intervention: Enabling prefix caching in LLM serving engines; varying weight quantization formats (16-bit and 4-bit); modifying server-level cache settings; restoring cache state.. Compared with: Caching disabled (cache-disabled runs); default cache behavior versus controlled cache state restoration.. n = 800. Not stated; presumed single research location..
Enabling cache changed agent trajectory on 36.2 percent of episodes at 16-bit precision and 75.0 percent at 4-bit quantization. With caching disabled, repeated execution was bit-identical across all 800 episodes in every configuration, bounding other nondeterminism sources at 0.5 percent. A single server-level prompt-cache setting moved run-to-run divergence by 37.5 percentage points, establishing a causal locus.
Study does not report whether divergence at task level affects downstream decision correctness or safety-critical outcomes.
This work is not clinical. For LLM system operators and developers: the finding suggests that prefix caching, commonly enabled by default for performance, introduces irreproducibility that scales with quantization—a common deployment choice. Practitioners relying on reproducibility or audit trails for agentic systems should be aware that cache state management is currently absent from request-level guarantees.
A single-site, uncontrolled empirical measurement of nondeterminism in LLM serving systems under specific configurations; demonstrates a real phenomenon but lacks comparative intervention, power calculation, or generalization across production workloads.
As stated by the source record.
Quoted from the source exactly as published.
This work is not clinical. For LLM system operators and developers: the finding suggests that prefix caching, commonly enabled by default for performance, introduces irreproducibility that scales with quantization—a common deployment choice. Practitioners relying on reproducibility or audit trails for agentic systems should be aware that cache state management is currently absent from request-level guarantees.
Graded across the dimensions that decide whether you should act, each from what the source actually supports. There is no single score, and where a dimension was not assessed it says so.
Prefix caching, in which a serving engine reuses the key and value tensors of a shared prompt prefix across requests, is enabled by default in the major open-source stacks and treated as a transparent optimization. We measure what it costs in reproducibility, and find that the cost rises sharply with weight quantization. Holding the model, decoding parameters, seed, and request order fixed, and issuing every request serially at batch size one, we ran an eighty-episode multi-turn agentic tool-use workload with caching enabled and disabled across two engines and four weight formats. Enabling the cache changed the agent's trajectory on 36.2 percent of episodes at 16-bit precision and on 75.0 percent at four-bit, a gradient that survives re-measurement under a controlled cache configuration. With caching disabled, repeated execution was bit-identical in every configuration, 0 of 800 episodes, which bounds other sources of nondeterminism at 0.5 percent. Repeated cache-enabled runs did diverge, and three experiments locate the cause: a single server-level prompt-cache setting moves run-to-run divergence by 37.5 percentage points, execution order acts only while that setting is active, and restoring cache state makes the cached and recompute paths each reproduce on 40 of 40 items while still differing from each other on 14. Cached serving is deterministic given cache state, and irreproducible in practice because that state is absent from the request and never reset by default. A single-turn bridge shows the divergence reaching task outcomes without shifting aggregate accuracy. We release the harness, logs, and analysis pipeline.
Taken from the source record, never inferred. Follow any of these and new work involving them reaches your briefing.