Abstract:Training deep neural networks with noise and data heterogeneity is a major challenge. We introduce Lightweight Learnable Adaptive Weighting (LiLAW), a method that dynamically adjusts the loss weight of each training sample based on its evolving difficulty, categorized as easy, moderate, and hard, using only three global learnable scalar parameters. LiLAW learns to adaptively prioritize samples by updating these parameters with a single gradient descent step on a validation mini-batch after each training mini-batch, without requiring a clean, unbiased validation set. Experiments across general and medical imaging datasets, several noise types and levels, loss functions, and architectures with and without pretraining, including linear probing and full fine-tuning, show that LiLAW consistently improves accuracy and AUROC, especially in higher-noise settings, without requiring excessive tuning. We also obtain state-of-the-art results incorporating synthetic and augmented data from SynPAIN, GAITGen, ECG5000, and improved fairness on the Adult dataset. LiLAW is lightweight, practical, and computationally efficient, making it an effective, scalable approach to boost generalization and robustness across diverse deep learning training setups, especially in resource-constrained settings.
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2509.20786 [cs.LG] |
| (or arXiv:2509.20786v4 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2509.20786 arXiv-issued DOI via DataCite |
From: Abhishek Moturu [view email]
[v1]
Thu, 25 Sep 2025 06:13:25 UTC (4,187 KB)
[v2]
Fri, 13 Mar 2026 14:51:03 UTC (3,985 KB)
[v3]
Mon, 11 May 2026 17:58:54 UTC (6,151 KB)
[v4]
Wed, 13 May 2026 15:28:04 UTC (6,143 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。