惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

阮一峰的网络日志
阮一峰的网络日志
J
Java Code Geeks
Martin Fowler
Martin Fowler
宝玉的分享
宝玉的分享
V
Visual Studio Blog
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
M
MIT News - Artificial intelligence
U
Unit 42
博客园 - 三生石上(FineUI控件)
博客园 - 聂微东
The GitHub Blog
The GitHub Blog
I
InfoQ
WordPress大学
WordPress大学
H
Help Net Security
D
Docker
B
Blog
腾讯CDC
A
About on SuperTechFans
Recent Announcements
Recent Announcements
雷峰网
雷峰网
有赞技术团队
有赞技术团队
C
Check Point Blog
Y
Y Combinator Blog
Cyber Security Advisories - MS-ISAC
Cyber Security Advisories - MS-ISAC

stat.ML updates on arXiv.org

Adaptive multi-fidelity optimization with fast learning rates Enhancing AI and Dynamical Subseasonal Forecasts with Probabilistic Bias Correction Sample Complexity Bounds for Stochastic Shortest Path with a Generative Model The Harder Path: Last Iterate Convergence for Uncoupled Learning in Zero-Sum Games with Bandit Feedback Stylistic-STORM (ST-STORM) : Perceiving the Semantic Nature of Appearance Collective Kernel EFT for Pre-activation ResNets PRIM-cipal components analysis One-Shot Generative Flows: Existence and Obstructions Structural interpretability in SVMs with truncated orthogonal polynomial kernels Amortized Optimal Transport from Sliced Potentials MinShap: A Modified Shapley Value Approach for Feature Selection Unsupervised feature selection using Bayesian Tucker decomposition Multi-User mmWave Beam and Rate Adaptation via Combinatorial Satisficing Bandits Best of both worlds: Stochastic & adversarial best-arm identification Scalable Model-Based Clustering with Sequential Monte Carlo Expert-Guided Class-Conditional Goodness-of-Fit Scores for Interpretable Classification with Informative Missingness: An Application to Seismic Monitoring Lightweight Geometric Adaptation for Training Physics-Informed Neural Networks Gating Enables Curvature: A Geometric Expressivity Gap in Attention Zeroth-Order Optimization at the Edge of Stability Differentially Private Conformal Prediction CLion: Efficient Cautious Lion Optimizer with Enhanced Generalization Generative Augmented Inference Improving Machine Learning Performance with Synthetic Augmentation PAC-MCTS: Bias-Aware Pruning for Robust LLM-Guided Search and Planning Path-Sampled Integrated Gradients Heat and Matérn Kernels on Matchings Doubly Outlier-Robust Online Infinite Hidden Markov Model Momentum Further Constrains Sharpness at the Edge of Stochastic Stability Multistage Conditional Compositional Optimization BOAT: Navigating the Sea of In Silico Predictors for Antibody Design via Multi-Objective Bayesian Optimization
Unified Analysis of Stochastic Gradient Methods for Compo...
Ahmed Khaled, Othmane Sebbouh, Nicolas Loizou, Robert M. Gower, · 2020-06-20 · via stat.ML updates on arXiv.org

We present a unified theorem for the convergence analysis of stochastic gradient algorithms for minimizing a smooth and convex loss plus a convex regularizer. We do this by extending the unified analysis of Gorbunov, Hanzely \& Richtárik (2020) and dropping the requirement that the loss function be strongly convex. Instead, we only rely on convexity of the loss function. Our unified analysis applies to a host of existing algorithms such as proximal SGD, variance reduced methods, quantization and some coordinate descent type methods. For the variance reduced methods, we recover the best known convergence rates as special cases. For proximal SGD, the quantization and coordinate type methods, we uncover new state-of-the-art convergence rates. Our analysis also includes any form of sampling and minibatching. As such, we are able to determine the minibatch size that optimizes the total complexity of variance reduced methods. We showcase this by obtaining a simple formula for the optimal minibatch size of two variance reduced methods (\textit{L-SVRG} and \textit{SAGA}). This optimal minibatch size not only improves the theoretical total complexity of the methods but also improves their convergence in practice, as we show in several experiments.