Abstract:To protect privacy in regulated domains such as healthcare and finance, model owners may allow only remote API access while keeping both the training data and model parameters private. However, model users performing inference on such remotely hosted models may be required to transmit potentially sensitive inputs, raising privacy concerns. In this work, we present LDPKiT, a framework for non-adversarial, privacy-preserving model distillation that leverages a user's private in-distribution data while bounding privacy leakage. LDPKiT introduces a novel superimposition technique that generates approximately in-distribution samples, enabling effective knowledge transfer under local differential privacy (LDP). Experiments on Fashion-MNIST, SVHN, and PathMNIST demonstrate that LDPKiT consistently improves utility while maintaining privacy, with benefits that become more pronounced at stronger noise levels. For example, on SVHN, LDPKiT achieves nearly the same inference accuracy at $\epsilon=1.25$ as at $\epsilon=2.0$, yielding stronger privacy guarantees with less than a 2\% accuracy reduction. We further conduct sensitivity analyses to examine the effect of dataset size on performance and provide a systematic analysis of latent space representations, offering intuitive and empirical insights into the accuracy gains of LDPKiT.
From: Kexin Li [view email]
[v1]
Sat, 25 May 2024 21:53:58 UTC (1,352 KB)
[v2]
Thu, 23 Jan 2025 19:41:02 UTC (45,498 KB)
[v3]
Sat, 11 Oct 2025 20:01:53 UTC (18,666 KB)
[v4]
Fri, 10 Jul 2026 10:17:50 UTC (7,636 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。