


























We study the relative entropy of the empirical probability vector with respect to the true probability vector in multinomial sampling of $k$ categories, which, when multiplied by sample size $n$, is also the log-likelihood ratio statistic. We generalize a recent result and show that the moment generating function of the statistic is bounded by a polynomial of degree $n$ on the unit interval, uniformly over all true probability vectors. We characterize the family of polynomials indexed by $(k,n)$ and obtain explicit formulae. Consequently, we develop Chernoff-type tail bounds, including a closed-form version from a large sample expansion of the bound minimizer. Our bound dominates the classic method-of-types bound and is competitive with the state of the art. We demonstrate with an application to estimating the proportion of unseen butterflies.
此内容由惯性聚合(RSS阅读器)自动聚合整理,仅供阅读参考。 原文来自 — 版权归原作者所有。