Abstract:We study a quantity called discrete layered entropy, which approximates the Shannon entropy within a logarithmic gap. Compared to the Shannon entropy, the discrete layered entropy is piecewise linear, approximates the expected length of the optimal one-to-one non-prefix code, and satisfies an elegant conditioning property. These properties make it useful for approximating the Shannon entropy in linear programming and maximum entropy problems, studying the optimal length of conditional encoding, and bounding the entropy of monotonic mixture distributions. In particular, it can give a bound $I(X;Y)+\log(I(X;Y)+3.4)+1$ for the strong functional representation lemma which is optimal within $2.8$ bits, and significantly improves upon the best known bound.
From: Cheuk Ting Li [view email]
[v1]
Thu, 23 Jan 2025 15:12:11 UTC (282 KB)
[v2]
Tue, 10 Jun 2025 20:19:03 UTC (2,146 KB)
[v3]
Sun, 25 Jan 2026 19:32:06 UTC (1,921 KB)
[v4]
Mon, 27 Jul 2026 16:29:59 UTC (1,892 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。