Abstract:Concept erasure aims to remove unwanted attributes, such as social or demographic factors, from learned representations, while preserving their task-relevant utility. While the goal of concept erasure is protection against all adversaries, existing methods remain vulnerable to nonlinear ones. This vulnerability arises from their failure to fully capture the complex, nonlinear statistical dependencies between learned representations and unwanted attributes. Moreover, although the existence of a trade-off between utility and erasure is expected, its progression during the erasure process, i.e., the cost of erasure, remains unstudied. In this work, we introduce Obliviator, a post-hoc erasure method designed to fully capture nonlinear statistical dependencies. We formulate erasure from a functional perspective, leading to an optimization problem involving a composition of kernels that lacks a closed-form solution. Instead of solving this problem in a single shot, we adopt an iterative approach that gradually morphs the feature space to achieve a more utility-preserving erasure. Unlike prior methods, Obliviator guards unwanted attribute against nonlinear adversaries. Our gradual approach quantifies the cost of nonlinear guardedness and reveals the dynamics between attribute protection and utility-preservation over the course of erasure. The utility-erasure trade-off curves obtained by Obliviator outperform the baselines and demonstrate its strong generalizability: its erasure becomes more utility-preserving when applied to the better-disentangled representations learned by more capable models.
| Comments: | Accepted to NeurIPS 2025 [Poster]. Code available at: this https URL |
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2603.07529 [cs.LG] |
| (or arXiv:2603.07529v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2603.07529 arXiv-issued DOI via DataCite |
|
| Journal reference: | The Thirty-Ninth Annual Conference on Neural Information Processing Systems 2025 |
From: Ramin Akbari [view email]
[v1]
Sun, 8 Mar 2026 08:35:43 UTC (3,393 KB)
[v2]
Wed, 29 Apr 2026 16:35:56 UTC (3,406 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。