Abstract:We demonstrate that learning procedures that rely on aggregated labels, e.g., label information distilled from noisy responses, enjoy robustness properties impossible without data cleaning. This robustness appears in several ways. In the context of risk consistency -- when one takes the standard approach in machine learning of minimizing a surrogate (typically convex) loss in place of a desired task loss (such as the zero-one mis-classification error) -- procedures using label aggregation obtain stronger consistency guarantees than those even possible using raw labels. And while classical statistical scenarios of fitting perfectly-specified models suggest that incorporating all possible information -- modeling uncertainty in labels -- is statistically efficient, consistency fails for ``standard'' approaches as soon as a loss to be minimized is even slightly mis-specified. Yet procedures leveraging aggregated information still converge to optimal classifiers, highlighting how incorporating a fuller view of the data analysis pipeline, from collection to model-fitting to prediction time, can yield a more robust methodology by refining noisy signals.
| Comments: | 41 pages, 3 figures. Accepted to Transactions on Machine Learning Research (TMLR) |
| Subjects: | Machine Learning (stat.ML); Machine Learning (cs.LG); Statistics Theory (math.ST) |
| Cite as: | arXiv:2509.11379 [stat.ML] |
| (or arXiv:2509.11379v3 [stat.ML] for this version) | |
| https://doi.org/10.48550/arXiv.2509.11379 arXiv-issued DOI via DataCite |
From: Chen Cheng [view email]
[v1]
Sun, 14 Sep 2025 18:17:51 UTC (53 KB)
[v2]
Wed, 28 Jan 2026 01:01:25 UTC (405 KB)
[v3]
Fri, 22 May 2026 19:53:27 UTC (1,038 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。