Loading
Life sciences · Preprint
arXiv · September 10, 2026
Early or partial results. Treat as a signal, not a conclusion.
This preprint describes a phase-decoupled power control algorithm for disaggregated LLM serving that the authors report achieves +20.4% energy efficiency (tokens/J) with +3.5% latency overhead on Qwen3-Coder-480B, compared to +8.6% efficiency at +5.2% latency cost for NVIDIA's Max-Q profile. The work is empirically grounded in production hardware and identifies a real problem (single power profile across heterogeneous prefill/decode lanes), but has not undergone peer review, results are limited to mixture-of-experts models, and lacks independent replication.
Empirical systems evaluation with A/B comparison. NVIDIA B200 disaggregated LLM serving cluster running Qwen3-Coder-480B (FP8 quantization) and Qwen3-235B-A22B (NVFP4 quantization) under agentic load pattern.. Intervention: Phase-decoupled, model-calibrated power control with latency-gated calibration under SLO guard; prefill SM-clock window and decode power cap placement.. Compared with: NVIDIA Max-Q inference profile; static clock locks; vendor default profiles..
NVIDIA Max-Q on disaggregated B200 system: +8.6% tokens/J gain with +5.2% mean end-to-end latency cost Proposed phase-decoupled controller on 8x B200 serving Qwen3-Coder-480B (FP8): +20.4% tokens/J at +3.5% mean e2e latency On Qwen3-235B-A22B (NVFP4): proposed controller meets ITL-p99 SLO in every repetition; both vendor profiles miss it
Safety was not reported in the material analysed. Check the source before drawing any conclusion about harm.
The source did not state who this applies to in practice.
This is a systems engineering study presenting a novel power control algorithm for LLM serving with reported empirical results on specific hardware and models, but it is a preprint lacking peer review, lacks independent validation, and claims are scoped to mixture-of-experts models only.
As stated by the source record.
Quoted from the source exactly as published.
Graded across the dimensions that decide whether you should act, each from what the source actually supports. There is no single score, and where a dimension was not assessed it says so.
Datacenter GPU power is the binding constraint on LLM serving capacity, and production serving has shifted to prefill/decode (PD) disaggregation. Deploying NVIDIA's Max-Q inference profile on a disaggregated B200 system, we found its realized gain modest (+8.6% tokens/J), model-dependent, and carrying a mean end-to-end latency cost (+5.2%) that throughput-only evaluation does not surface; the profile also applies one setting to prefill and decode GPUs that operate in opposite hardware regimes. We hypothesize that the optimal power setting is a property of the deployed (model, quantization, engine, hardware) combination rather than of the GPU class, that each lane warrants its own profile, and that converting SLO headroom into energy safely requires latency-gated calibration under a runtime SLO guard rather than a fixed recipe. We present a phase-decoupled, model-calibrated controller: the prefill lane runs under an SM-clock window whose floor is a latency guarantee by construction, and the decode lane under a power cap placed by automatic calibration just above a measured throughput/latency cliff. Because a disaggregated decode lane draws flat, memory-bound power, the cap binds continuously, the reactive-overshoot weakness that led POLCA to reject capping is absent, and the GPU's own power manager retains throughput under the cap. On an 8x B200 node serving Qwen3-Coder-480B (FP8) under agentic load, our balanced mode delivers +20.4% tokens/J at +3.5% mean e2e versus +8.6% at +5.2% for Max-Q, a Pareto improvement on both axes. On Qwen3-235B-A22B (NVFP4) every operating mode meets the ITL-p99 SLO in every repetition; both vendor profiles miss it. A decode-actuator A/B shows the calibrated cap beats static clock locks, and a three-day sustained run saves 32.3% of a lane pair's electricity. Both models are MoE; a dense model recovers roughly 5x less, so we scope our claims to MoE serving.
Taken from the source record, never inferred. Follow any of these and new work involving them reaches your briefing.