Abstract:Recent efforts to develop trustworthy AI systems have increased interest in learning problems with explicit requirements, or constraints. In deep learning, however, such problems are often handled through fixed weighted-sum penalization: the constraints are added to the task loss with fixed coefficients, and the resulting scalarized objective is minimized. This position paper argues that fixed penalization is often ill-suited for deep learning problems with non-negotiable requirements for several reasons. First, in non-convex settings, the penalized and constrained problems are generally not equivalent, so solving the former need not solve the latter. Second, fixed penalization weakens hard requirements into soft penalties to be traded off against task performance. Third, choosing penalty coefficients to indirectly solve the constrained problem often involves costly trial and error, because changing them alters the penalized objective itself, and hence can mean solving the wrong problem altogether. We therefore argue that, when a deep learning problem specifies non-negotiable requirements, the constrained formulation itself should be the starting point, not the surrogate problem defined by fixed penalization. The appropriate solution strategy should then be chosen based on the problem's structure and scale.
| Comments: | Code available at this https URL |
| Subjects: | Machine Learning (cs.LG); Optimization and Control (math.OC) |
| Cite as: | arXiv:2505.20628 [cs.LG] |
| (or arXiv:2505.20628v4 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2505.20628 arXiv-issued DOI via DataCite |
From: Meraj Hashemizadeh [view email]
[v1]
Tue, 27 May 2025 02:09:17 UTC (302 KB)
[v2]
Wed, 9 Jul 2025 19:47:30 UTC (399 KB)
[v3]
Mon, 28 Jul 2025 19:38:14 UTC (398 KB)
[v4]
Thu, 7 May 2026 00:19:08 UTC (384 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。