惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

WordPress大学
WordPress大学
A
About on SuperTechFans
小众软件
小众软件
Hugging Face - Blog
Hugging Face - Blog
奇客Solidot–传递最新科技情报
奇客Solidot–传递最新科技情报
博客园 - 叶小钗
博客园 - 聂微东
博客园 - Franky
Apple Machine Learning Research
Apple Machine Learning Research
罗磊的独立博客
量子位
博客园 - 三生石上(FineUI控件)
Recent Announcements
Recent Announcements
The GitHub Blog
The GitHub Blog
B
Blog RSS Feed
T
The Blog of Author Tim Ferriss
GbyAI
GbyAI
云风的 BLOG
云风的 BLOG
Last Week in AI
Last Week in AI
宝玉的分享
宝玉的分享
B
Blog
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
Stack Overflow Blog
Stack Overflow Blog
Cyber Security Advisories - MS-ISAC
Cyber Security Advisories - MS-ISAC

stat.ML updates on arXiv.org

Adaptive multi-fidelity optimization with fast learning rates Enhancing AI and Dynamical Subseasonal Forecasts with Probabilistic Bias Correction Sample Complexity Bounds for Stochastic Shortest Path with a Generative Model The Harder Path: Last Iterate Convergence for Uncoupled Learning in Zero-Sum Games with Bandit Feedback Stylistic-STORM (ST-STORM) : Perceiving the Semantic Nature of Appearance Collective Kernel EFT for Pre-activation ResNets PRIM-cipal components analysis One-Shot Generative Flows: Existence and Obstructions Structural interpretability in SVMs with truncated orthogonal polynomial kernels Amortized Optimal Transport from Sliced Potentials MinShap: A Modified Shapley Value Approach for Feature Selection Unsupervised feature selection using Bayesian Tucker decomposition Multi-User mmWave Beam and Rate Adaptation via Combinatorial Satisficing Bandits Best of both worlds: Stochastic & adversarial best-arm identification Scalable Model-Based Clustering with Sequential Monte Carlo Expert-Guided Class-Conditional Goodness-of-Fit Scores for Interpretable Classification with Informative Missingness: An Application to Seismic Monitoring Lightweight Geometric Adaptation for Training Physics-Informed Neural Networks Gating Enables Curvature: A Geometric Expressivity Gap in Attention Zeroth-Order Optimization at the Edge of Stability Differentially Private Conformal Prediction CLion: Efficient Cautious Lion Optimizer with Enhanced Generalization Generative Augmented Inference Improving Machine Learning Performance with Synthetic Augmentation PAC-MCTS: Bias-Aware Pruning for Robust LLM-Guided Search and Planning Path-Sampled Integrated Gradients Heat and Matérn Kernels on Matchings Doubly Outlier-Robust Online Infinite Hidden Markov Model Momentum Further Constrains Sharpness at the Edge of Stochastic Stability Multistage Conditional Compositional Optimization BOAT: Navigating the Sea of In Silico Predictors for Antibody Design via Multi-Objective Bayesian Optimization
Optimal Sketching Bounds for Sparse Linear Regression
Tung Mai, Alexander Munteanu, Cameron Musco, Anup B. Rao, Chris · 2023-04-05 · via stat.ML updates on arXiv.org

We study oblivious sketching for $k$-sparse linear regression under various loss functions such as an $\ell_p$ norm, or from a broad class of hinge-like loss functions, which includes the logistic and ReLU losses. We show that for sparse $\ell_2$ norm regression, there is a distribution over oblivious sketches with $Θ(k\log(d/k)/\varepsilon^2)$ rows, which is tight up to a constant factor. This extends to $\ell_p$ loss with an additional additive $O(k\log(k/\varepsilon)/\varepsilon^2)$ term in the upper bound. This establishes a surprising separation from the related sparse recovery problem, which is an important special case of sparse regression. For this problem, under the $\ell_2$ norm, we observe an upper bound of $O(k \log (d)/\varepsilon + k\log(k/\varepsilon)/\varepsilon^2)$ rows, showing that sparse recovery is strictly easier to sketch than sparse regression. For sparse regression under hinge-like loss functions including sparse logistic and sparse ReLU regression, we give the first known sketching bounds that achieve $o(d)$ rows showing that $O(μ^2 k\log(μn d/\varepsilon)/\varepsilon^2)$ rows suffice, where $μ$ is a natural complexity parameter needed to obtain relative error bounds for these loss functions. We again show that this dimension is tight, up to lower order terms and the dependence on $μ$. Finally, we show that similar sketching bounds can be achieved for LASSO regression, a popular convex relaxation of sparse regression, where one aims to minimize $\|Ax-b\|_2^2+λ\|x\|_1$ over $x\in\mathbb{R}^d$. We show that sketching dimension $O(\log(d)/(λ\varepsilon)^2)$ suffices and that the dependence on $d$ and $λ$ is tight.