Abstract:Federated Large Language Models (FedLLMs) enable multiple parties to collaboratively fine-tune LLMs without sharing raw data, addressing challenges of limited resources and privacy concerns. Despite data localization, shared gradients can still expose sensitive information through membership inference attacks (MIAs). However, FedLLMs' unique properties, i.e. massive parameter scales, rapid convergence, and sparse, non-orthogonal gradients, render existing MIAs ineffective. To address this gap, we propose ProjRes, the first projection residuals-based passive MIA tailored for FedLLMs. ProjRes leverages hidden embedding vectors as sample representations and analyzes their projection residuals on the gradient subspace to uncover the intrinsic link between gradients and inputs. It requires no shadow models, auxiliary classifiers, or historical updates, ensuring efficiency and robustness. Experiments on four benchmarks and four LLMs show that ProjRes achieves near 100% accuracy, outperforming prior methods by up to 75.75%, and remains effective even under strong differential privacy defenses. Our findings reveal a previously overlooked privacy vulnerability in FedLLMs and call for a re-examination of their security assumptions. Our code and data are available at $\href{this https URL}{link}$.
| Comments: | This is the full version (including complete appendices and supplementary materials) of the paper accepted for publication at the 2026 IEEE Symposium on Security and Privacy |
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2604.21197 [cs.LG] |
| (or arXiv:2604.21197v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2604.21197 arXiv-issued DOI via DataCite (pending registration) |
From: Silong Chen [view email]
[v1]
Thu, 23 Apr 2026 01:44:04 UTC (1,481 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。