Abstract:The LogSumExp function, dual to the Kullback-Leibler (KL) divergence, plays a central role in many important optimization problems, including entropy-regularized optimal transport (OT) and distributionally robust optimization (DRO). In practice, when the number of exponential terms inside the logarithm is large or infinite, optimization becomes challenging since computing the gradient requires differentiating every term. We propose a novel convexity- and smoothness-preserving approximation to LogSumExp that can be efficiently optimized using stochastic gradient methods. This approximation is rooted in a sound modification of the KL divergence in the dual, resulting in a new $f$-divergence called the Safe KL divergence. Our experiments and theoretical analysis of the LogSumExp-based stochastic optimization, arising in DRO and continuous OT, demonstrate the advantages of our approach over existing baselines.
From: Egor Gladin [view email]
[v1]
Mon, 29 Sep 2025 15:03:55 UTC (1,884 KB)
[v2]
Thu, 4 Dec 2025 11:38:28 UTC (2,123 KB)
[v3]
Tue, 3 Feb 2026 14:47:43 UTC (1,566 KB)
[v4]
Thu, 18 Jun 2026 13:27:17 UTC (1,577 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。