惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

云风的 BLOG
云风的 BLOG
GbyAI
GbyAI
G
Google Developers Blog
Engineering at Meta
Engineering at Meta
月光博客
月光博客
腾讯CDC
Recent Announcements
Recent Announcements
酷 壳 – CoolShell
酷 壳 – CoolShell
爱范儿
爱范儿
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
S
SegmentFault 最新的问题
奇客Solidot–传递最新科技情报
奇客Solidot–传递最新科技情报
阮一峰的网络日志
阮一峰的网络日志
博客园 - 【当耐特】
The GitHub Blog
The GitHub Blog
Last Week in AI
Last Week in AI
让小产品的独立变现更简单 - ezindie.com
让小产品的独立变现更简单 - ezindie.com
aimingoo的专栏
aimingoo的专栏
Google DeepMind News
Google DeepMind News
Y
Y Combinator Blog
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
Martin Fowler
Martin Fowler
A
About on SuperTechFans
博客园 - 叶小钗

stat.ML updates on arXiv.org

Adaptive multi-fidelity optimization with fast learning rates Enhancing AI and Dynamical Subseasonal Forecasts with Probabilistic Bias Correction Sample Complexity Bounds for Stochastic Shortest Path with a Generative Model The Harder Path: Last Iterate Convergence for Uncoupled Learning in Zero-Sum Games with Bandit Feedback Stylistic-STORM (ST-STORM) : Perceiving the Semantic Nature of Appearance Collective Kernel EFT for Pre-activation ResNets PRIM-cipal components analysis One-Shot Generative Flows: Existence and Obstructions Structural interpretability in SVMs with truncated orthogonal polynomial kernels Amortized Optimal Transport from Sliced Potentials MinShap: A Modified Shapley Value Approach for Feature Selection Unsupervised feature selection using Bayesian Tucker decomposition Multi-User mmWave Beam and Rate Adaptation via Combinatorial Satisficing Bandits Best of both worlds: Stochastic & adversarial best-arm identification Scalable Model-Based Clustering with Sequential Monte Carlo Expert-Guided Class-Conditional Goodness-of-Fit Scores for Interpretable Classification with Informative Missingness: An Application to Seismic Monitoring Lightweight Geometric Adaptation for Training Physics-Informed Neural Networks Gating Enables Curvature: A Geometric Expressivity Gap in Attention Zeroth-Order Optimization at the Edge of Stability Differentially Private Conformal Prediction CLion: Efficient Cautious Lion Optimizer with Enhanced Generalization Generative Augmented Inference Improving Machine Learning Performance with Synthetic Augmentation PAC-MCTS: Bias-Aware Pruning for Robust LLM-Guided Search and Planning Path-Sampled Integrated Gradients Heat and Matérn Kernels on Matchings Doubly Outlier-Robust Online Infinite Hidden Markov Model Momentum Further Constrains Sharpness at the Edge of Stochastic Stability Multistage Conditional Compositional Optimization BOAT: Navigating the Sea of In Silico Predictors for Antibody Design via Multi-Objective Bayesian Optimization
Improved Global Guarantees for the Nonconvex Burer--Monte...
Richard Y. Zhang · 2022-07-05 · via stat.ML updates on arXiv.org

We consider minimizing a twice-differentiable, $L$-smooth, and $μ$-strongly convex objective $φ$ over an $n\times n$ positive semidefinite matrix $M\succeq0$, under the assumption that the minimizer $M^{\star}$ has low rank $r^{\star}\ll n$. Following the Burer--Monteiro approach, we instead minimize the nonconvex objective $f(X)=φ(XX^{T})$ over a factor matrix $X$ of size $n\times r$. This substantially reduces the number of variables from $O(n^{2})$ to as few as $O(n)$ and also enforces positive semidefiniteness for free, but at the cost of giving up the convexity of the original problem. In this paper, we prove that if the search rank $r\ge r^{\star}$ is overparameterized by a \emph{constant factor} with respect to the true rank $r^{\star}$, namely as in $r>\frac{1}{4}(L/μ-1)^{2}r^{\star}$, then despite nonconvexity, local optimization is guaranteed to globally converge from any initial point to the global optimum. This significantly improves upon a previous rank overparameterization threshold of $r\ge n$, which we show is sharp in the absence of smoothness and strong convexity, but would increase the number of variables back up to $O(n^{2})$. Conversely, without rank overparameterization, we prove that such a global guarantee is possible if and only if $φ$ is almost perfectly conditioned, with a condition number of $L/μ<3$. Therefore, we conclude that a small amount of overparameterization can lead to large improvements in theoretical guarantees for the nonconvex Burer--Monteiro factorization.