





















Abstract:Choosing between classical and Bayesian sparse regression methods involves a real trade-off: penalized estimators like Lasso run in milliseconds but give no uncertainty estimates,while Horseshoe and Spike-and-Slab priors produce full posteriors but need MCMC chains that take minutes per this http URL few studies compare these two families head-to-head under the conditions that actually make sparse regression hard -- correlated features, weak signals, and growing dimensionality. We benchmark six methods (OLS, Ridge,Lasso, Elastic Net, Horseshoe, Spike-and-Slab) on synthetic data with three covariance structures (rho up to 0.9), four SNR levels, and p in {20, 50, 100}, plus the Diabetes dataset,totalling over 2,600 experiments. The results are clear on some points and nuanced on others. Bayesian methods win on prediction error (MSE 72 vs. 108-267), and the Horseshoe delivers near-nominal 95% coverage (94.8%). But Spike-and-Slab,despite narrower intervals, under-covers at 91.9% -- its continuous relaxation likely plays a role. For variable selection, Lasso and Spike-and-Slab tie at F1 ~ 0.47, making Lasso the practical default when posteriors are not needed. Code and data are available at this https URL.
| Comments: | 14 pages, 8 figures, 6 tables. Code: this https URL |
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2605.00835 [cs.LG] |
| (or arXiv:2605.00835v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2605.00835 arXiv-issued DOI via DataCite |
From: Hao Xiao [view email]
[v1]
Sat, 4 Apr 2026 15:46:44 UTC (71 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理,仅供阅读参考。 原文来自 — 版权归原作者所有。