










Abstract:GPU sharing faces a determinism--utilization tradeoff: fixed bindings can strand capacity as demand fluctuates, while resource-driven kernel reshaping improves utilization by altering a launch's parallel structure, potentially changing output bits. We rethink modern GPU scheduling and observe that it decouples logical structure from physical width: one unmodified launch spans a family of widths through changes in block placement and wave count. From this observation, we derive the parallel-structure invariant: for fixed-structure deterministic workloads, keeping each launch immutable makes its output bits independent of physical width.
Guided by this invariant, Vitamin-E late-binds immutable launches to pooled physical contexts, preserving bitwise equality across allocations, whereas resource-driven reshaping can alter the selected token under temperature-zero greedy decoding. Across all workload--baseline comparisons, Vitamin-E achieves up to 3.50$\times$ the aggregate normalized LLM training throughput, 62.5\% lower inference p99 latency, and 1.43$\times$ the background-training throughput. With the same mechanism, \textsc{TPOT-First} reduces TPOT SLO violations by up to 46.1\% over \textsc{Throughput-Oriented} on three serving workloads, demonstrating mechanism effectiveness and policy flexibility.
From: Zhenyuan Yang [view email]
[v1]
Mon, 16 Mar 2026 09:48:34 UTC (3,121 KB)
[v2]
Tue, 17 Mar 2026 08:51:40 UTC (3,121 KB)
[v3]
Fri, 3 Apr 2026 11:51:07 UTC (3,212 KB)
[v4]
Wed, 5 Aug 2026 05:18:06 UTC (796 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理,仅供阅读参考。 原文来自 — 版权归原作者所有。