惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

博客园 - 三生石上(FineUI控件)
U
Unit 42
人人都是产品经理
人人都是产品经理
罗磊的独立博客
Recent Announcements
Recent Announcements
云风的 BLOG
云风的 BLOG
aimingoo的专栏
aimingoo的专栏
T
Tailwind CSS Blog
GbyAI
GbyAI
Blog — PlanetScale
Blog — PlanetScale
I
InfoQ
Last Week in AI
Last Week in AI
宝玉的分享
宝玉的分享
B
Blog RSS Feed
WordPress大学
WordPress大学
腾讯CDC
H
Help Net Security
博客园 - Franky
博客园 - 【当耐特】
博客园 - 聂微东
Stack Overflow Blog
Stack Overflow Blog
B
Blog
Vercel News
Vercel News
博客园 - 司徒正美

math.PR updates on arXiv.org

Visibility in the Boolean Model on Harmonic Manifolds Global estimates on the Brenier map Geodesics and Wandering Exponents in Brochette First-Passage Percolation State-dependent inverse-subordinator time changes of regenerative processes: Excursion structure and multiscale occupation-time limits Randomly twisted transfer operators and singular values statistics Generalized Bessel-Dunkl diffusions An almost sure invariance principle for the Takagi-van der Waerden class functions Central limit theorems for high dimensional lattice polytopes: cosmological polytopes Convergence rate estimates for semigroups and heat kernels associated with resistance forms Second-order Poincaré inequalities and localization on the Poisson space Maximum Probability of Independence in Transitive Matroids On global solutions to the semidiscrete stochastic heat equation The Poisson Tail Conjecture for primes in short intervals A Complete Spectral Analysis of the CEV Operator with Applications to Arbitrage Holographic functions and neural networks From Betting to Empirical Bernstein LIL Concentration of General Stochastic Approximation Under Heavy-Tailed Markovian Noise Pointwise Generalization in Deep Neural Networks Bayesian Latent Space Models for Graphs Are Misspecified: Toward Robust Inference via Generalized Posteriors Wasserstein bounds for denoising diffusion probabilistic models via the Föllmer process A note on connections between the Föllmer process and the denoising diffusion probabilistic model Simple Approximation and Derivative Free Inference-Time Scaling for Diffusion Models via Sequential Monte Carlo on Path Measures Diffusion-Based Stochastic Operator Networks for Uncertainty Quantification in Stochastic Partial Differential Equations A Fourier perspective on the learning dynamics of neural networks: from sample complexities to mechanistic insights Propagation of Chaos in Contextual Flow Maps Dimension-Uniform Discretization Analysis of Preconditioned Annealed Langevin Dynamics for Multimodal Gaussian Mixtures $α$-TCAV: A Unified Framework for Testing with Concept Activation Vectors Scaling Laws from Sequential Feature Recovery: A Solvable Hierarchical Model On the Limits of Latent Reuse in Diffusion Models State-of-art minibatches via novel DPP kernels: discretization, wavelets, and rough objectives
Complexity of Zeroth- and First-order Stochastic Trust-Re...
Yunsoo Ha, Sara Shashaani, Raghu Pasupathy · 2024-05-30 · via math.PR updates on arXiv.org

Model update (MU) and candidate evaluation (CE) are classical steps incorporated inside many stochastic trust-region (TR) algorithms. The sampling effort exerted within these steps, often decided with the aim of controlling model error, largely determines a stochastic TR algorithm's sample complexity. Given that MU and CE are amenable to variance reduction, we investigate the effect of incorporating common random numbers (CRN) within MU and CE on complexity. Using ASTRO and ASTRO-DF as prototype first-order and zeroth-order families of algorithms, we demonstrate that CRN's effectiveness leads to a range of complexities depending on sample-path regularity and the oracle order. For instance, we find that in first-order oracle settings with smooth sample paths, CRN's effect is pronounced -- ASTRO with CRN achieves $\tilde{O}(ε^{-2})$ a.s. sample complexity compared to $\tilde{O}(ε^{-6})$ a.s. in the generic no-CRN setting. By contrast, CRN's effect is muted when the sample paths are not Lipschitz, with the sample complexity improving from $\tilde{O}(ε^{-6})$ a.s. to $\tilde{O}(ε^{-5})$ and $\tilde{O}(ε^{-4})$ a.s. in the zeroth- and first-order settings, respectively. Since our results imply that improvements in complexity are largely inherited from generic aspects of variance reduction, e.g., finite-differencing for zeroth-order settings and sample-path smoothness for first-order settings within MU, we anticipate similar trends in other contexts.