Abstract:Machine unlearning is the task of updating a trained model to forget specific training data without retraining from scratch. In this paper, we investigate how unlearning of deep neural networks (DNNs) is affected by the model parameterization level, which corresponds here to the DNN width. We define validation-based tuning for several unlearning methods from the recent literature, and show how these methods perform differently depending on (i) the DNN parameterization level, (ii) the unlearning goal (unlearned data privacy or bias removal), (iii) whether the unlearning method explicitly uses the unlearned examples. Our results show that unlearning usually excels on overparameterized models by significantly improving privacy/bias at a reasonable cost of utility (generalization) degradation; although for bias removal this requires the unlearning method to use the unlearned examples. Furthermore, we measure how much the unlearning changes the classification decision regions in the proximity of the unlearned examples, and avoids changing them elsewhere. By this we show that the unlearning success for overparameterized models stems from the ability to delicately change the model functionality in small regions in the input space while keeping much of the model functionality unchanged.
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2503.08633 [cs.LG] |
| (or arXiv:2503.08633v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2503.08633 arXiv-issued DOI via DataCite |
From: Yehuda Dar [view email]
[v1]
Tue, 11 Mar 2025 17:21:26 UTC (9,947 KB)
[v2]
Tue, 19 May 2026 13:33:29 UTC (9,307 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。