惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

IT之家
IT之家
The GitHub Blog
The GitHub Blog
F
Fortinet All Blogs
Last Week in AI
Last Week in AI
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
L
LangChain Blog
爱范儿
爱范儿
博客园_首页
Stack Overflow Blog
Stack Overflow Blog
MongoDB | Blog
MongoDB | Blog
博客园 - 三生石上(FineUI控件)
大猫的无限游戏
大猫的无限游戏
宝玉的分享
宝玉的分享
GbyAI
GbyAI
H
Help Net Security
A
About on SuperTechFans
Recent Announcements
Recent Announcements
Hugging Face - Blog
Hugging Face - Blog
freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More
雷峰网
雷峰网
D
Docker
博客园 - Franky
有赞技术团队
有赞技术团队
G
Google Developers Blog

math.ST updates on arXiv.org

What is Learnable in Valiant's Theory of the Learnable? Learning Perturbations to Extrapolate Your LLM Byzantine-Robust Distributed Sparse Learning Revisited The Sample Complexity of Multiple Change Point Identification under Bandit Feedback A proximal gradient algorithm for composite log-concave sampling Model-based Bootstrap of Controlled Markov Chains Approximation of Maximally Monotone Operators : A Graph Convergence Perspective Posterior Contraction Rates for Sparse Kolmogorov-Arnold Networks in Anisotropic Besov Spaces MIST: Reliable Streaming Decision Trees for Online Class-Incremental Learning via McDiarmid Bound A Spectral Framework for Closed-Form Relative Density Estimation Fast Rates for Offline Contextual Bandits with Forward-KL Regularization under Single-Policy Concentrability Higher-Order Equilibrium Tracking for EM-Compressible Online Estimation Scaling Limits of Long-Context Transformers A Note on Non-Negative $L_1$-Approximating Polynomials Susceptibilities and Patterning: A Primer on Linear Response in Bayesian Learning Linear Response Estimators for Singular Statistical Models Statistical inference with belief functions: A survey Robust stochastic first order methods in heavy-tailed noise via medoid mini-batch gradient sampling Every Feedforward Neural Network Definable in an o-Minimal Structure Has Finite Sample Complexity Adaptive auditing of AI systems with anytime-valid guarantees Locally Near Optimal Piecewise Linear Regression in High Dimensions via Difference of Max-Affine Functions Risk-Controlled Post-Processing of Decision Policies Covariate Balancing and Riesz Regression Should Be Guided by the Neyman Orthogonal Score in Debiased Machine Learning A Unified Pair-GRPO Family: From Implicit to Explicit Preference Constraints for Stable and General RL Alignment Time-Inhomogeneous Preconditioned Langevin Dynamics A Fine-Grained Understanding of Uniform Convergence for Halfspaces CITE: Anytime-Valid Statistical Inference in LLM Self-Consistency Ratio-based Loss Functions Optimal Confidence Band for Kernel Gradient Flow Estimator A renormalization-group inspired lattice-based framework for piecewise generalized linear models
Optimal Covariance Estimation for Condition Number Loss i...
David L. Donoho, Behrooz Ghorbani · 2018-10-17 · via math.ST updates on arXiv.org

We study estimation of the covariance matrix under relative condition number loss $κ(Σ^{-1/2} \hatΣ Σ^{-1/2})$, where $κ(Δ)$ is the condition number of matrix $Δ$, and $\hatΣ$ and $Σ$ are the estimated and theoretical covariance matrices. Optimality in $κ$-loss provides optimal guarantees in two stylized applications: Multi-User Covariance Estimation and Multi-Task Linear Discriminant Analysis. We assume the so-called spiked covariance model for $Σ$, and exploit recent advances in understanding that model, to derive a nonlinear shrinker which is asymptotically optimal among orthogonally-equivariant procedures. In our asymptotic study, the number of variables $p$ is comparable to the number of observations $n$. The form of the optimal nonlinearity depends on the aspect ratio $γ=p/n$ of the data matrix and on the top eigenvalue of $Σ$. For $γ> 0.618...$, even dependence on the top eigenvalue can be avoided. The optimal shrinker has two notable properties. First, when $p/n \rightarrow γ\gg 1$ is large, it shrinks even very large eigenvalues substantially, by a factor $1/(1+γ)$. Second, even for moderate $γ$, certain highly statistically significant eigencomponents will be completely suppressed. We show that when $γ\gg 1$ is large, purely diagonal covariance matrices can be optimal, despite the top eigenvalues being large and the empirical eigenvalues being highly statistically significant. This aligns with practitioner experience. We identify intuitively reasonable procedures with small worst-case relative regret - the simplest being generalized soft thresholding having threshold at the bulk edge and slope $(1+γ)^{-1}$ above the bulk. For $γ< 2$ it has at most a few percent relative regret.