Abstract:AdaBoost sequentially fits so-called weak learners to minimize an exponential loss, which penalizes misclassified data points more severely than other loss functions like cross-entropy. Paradoxically, AdaBoost generalizes well in practice as the number of weak learners grows. In the present work, we introduce Penalized Exponential Loss (PENEX), a new formulation of the multi-class exponential loss that is theoretically grounded and, in contrast to the existing formulation, amenable to optimization via first-order methods, making it a practical objective for training neural networks. We demonstrate that PENEX effectively increases margins of data points, which can be translated into a generalization bound. Empirically, across computer vision and language tasks, PENEX improves neural network generalization in low-data regimes, matching and in some settings outperforming established regularizers at comparable computational cost. Our results highlight the potential of the exponential loss beyond its application in AdaBoost.
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2510.02107 [cs.LG] |
| (or arXiv:2510.02107v4 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2510.02107 arXiv-issued DOI via DataCite |
From: Klaus-Rudolf Kladny [view email]
[v1]
Thu, 2 Oct 2025 15:13:02 UTC (1,467 KB)
[v2]
Mon, 6 Oct 2025 17:51:59 UTC (1,467 KB)
[v3]
Fri, 30 Jan 2026 12:09:00 UTC (1,476 KB)
[v4]
Tue, 12 May 2026 09:50:01 UTC (1,409 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。