¹University of Illinois Urbana-Champaign ²Tsinghua University

Modern Large Language Model (LLM) serving systems increasingly support interactive applications such as real-time chat assistants, code generation tools, and agentic workflows. However, the soaring energy cost of LLM inference presents a growing challenge for sustainable and cost-effective deployment.

We introduce VoltanaLLM, a system for SLO-aware, energy-efficient LLM serving, designed from a control theory perspective. VoltanaLLM co-designs frequency scaling and request routing in emerging prefill/decode disaggregated architectures, leveraging their decoupled execution to enable fine-grained, phase-specific control. It consists of (1) a feedback-driven frequency controller that dynamically adapts GPU frequency for prefill and decode phases, and (2) a state-space router that explores routing decisions across frequency-scaled instances to minimize energy under latency constraints.

We implement VoltanaLLM in SGLang and evaluate its performance across multiple state-of-the-art LLMs and real-world datasets. Our results show that VoltanaLLM achieves up to 36.3% energy savings while maintaining a near-perfect SLO attainment rate, paving the way for sustainable and intelligent LLM serving.

Our empirical profiling of LLM inference reveals a non-monotonic energy–frequency relationship . As shown above, while reducing GPU frequency from 1410 MHz to 1005 MHz (by ~28.7%) does increase execution time, the increase is sub-linear. Consequently, the total energy follows a U-shaped curve with respect to GPU frequency. This trend indicates that at low frequencies, execution time dominates energy , whereas at high frequencies, power dominates ; in the middle lies an energy sweet point .

Numerous systems have been proposed to improve LLM serving efficiency. These include advanced batching strategies for throughput optimization, memory management techniques like PagedAttention, CPU offloading, and GPU kernel-level optimizations (e.g., FlashAttention). Parallelism frameworks and parameter-sharing mechanisms further reduce bottlenecks, while speculative decoding and preemptive scheduling improve tail latency and job completion times.

Several recent efforts have also begun addressing energy-efficient LLM serving. For instance, DynamoLLM explores GPU frequency control based on request characteristics, while μ-Serve optimizes power by co-serving multiple models. EcoServe considers operational and embodied carbon emissions, TAPAS exploits datacenter thermal slack, and Heron places GPUs closer to renewable sources.

To better manage compute heterogeneity, recent systems have introduced prefill/decode (P/D) disaggregation, separating the two phases across GPU nodes. Projects like SplitWise, TetriInfer, Llumnix, and DistServe show improvements in goodput, time-to-first-token, and SLO attainment. Popular inference libraries such as vLLM and SGLang have also added runtime support.

These efforts primarily optimize latency and throughput, but overlook the energy implications. P/D disaggregation creates unique opportunities for phase-specific frequency scaling and energy-aware routing—opportunities that VoltanaLLM systematically exploits.

These works highlight early signals for sustainable AI. However, they focus on coarse-grained control. VoltanaLLM is the first system to explore fine-grained frequency and routing control in prefill/decode disaggregated serving, with SLO-aware feedback loops.

VoltanaLLM runs on a Prefill/Decode (P/D) disaggregated serving stack and couples feedback-driven frequency control with state-space routing, steered by a lightweight latency predictor for SLO-aware, energy-efficient inference.

We evaluate VoltanaLLM on three models (Ministral-3B, LLaMA-3.1-8B, Qwen3-32B) and two real-world datasets under a 2P2D serving setup on NVIDIA A100 GPUs. Our results show that VoltanaLLM achieves energy savings without sacrificing latency SLO attainment:

📉 Energy Savings: Reduces GPU energy consumption by up to 36.3% while preserving SLOs for requests.

⚡ Latency SLOs: Maintains comparable TTFT/ITL SLO attainment to GPU serving set at maximum frequency (in this case 1410 MHz).

🔄 Adaptivity: Operates at low frequency when load permits, scaling up dynamically at higher request rates to sustain SLOs in near real-time with negligible overhead.

✅ Key Takeaway: VoltanaLLM enables SLO-aware, energy-efficient LLM inference, improving sustainability without compromising user experience.

  @misc{yu2025voltanallmfeedbackdrivenfrequencycontrol,
      title={VoltanaLLM: Feedback-Driven Frequency Control and State-Space Routing for Energy-Efficient LLM Serving}, 
      author={Jiahuan Yu and Aryan Taneja and Junfeng Lin and Minjia Zhang},
      year={2025},
      eprint={2509.04827},
      archivePrefix={arXiv},
      primaryClass={cs.DC},
      url={https://arxiv.org/abs/2509.04827}, 
}