Abstract:Smoothness is a key inductive bias in machine learning and is closely related to generalization. Existing smoothness-inducing methods typically rely either on explicit gradient regularization, which often incurs substantial computational and memory overhead, or on data-mixing strategies, which are less naturally applicable to unsupervised and self-supervised settings. In this work, we propose $\textit{Dual Reconstruction Smoothing}$ (DReS), a nonparametric regularization framework that induces smoothness through a spline-based auxiliary branch with shared model parameters. The method introduces no additional trainable parameters and can be applied to arbitrary submodules, making it suitable for unsupervised, self-supervised, and supervised regimes. We show theoretically that the discrepancy between the target function and its DReS approximation is controlled by higher-order smoothness quantities of the function, establishing the method as an implicit higher-order smoothness regularizer. Empirically, DReS improves representation learning across several self-supervised methods, improves generation quality in generative modeling, and achieves strong performance relative to competitive baselines in supervised learning.
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2510.00253 [cs.LG] |
| (or arXiv:2510.00253v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2510.00253 arXiv-issued DOI via DataCite |
From: Parsa Moradi [view email]
[v1]
Tue, 30 Sep 2025 20:24:48 UTC (16,288 KB)
[v2]
Fri, 8 May 2026 17:51:10 UTC (23,189 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。