Abstract:Many decentralized distillation methods are designed around training-time coordination, yet deploy each node in isolation even when more capable neighbors remain available at inference time. This is an incomplete objective for settings such as IoT, where devices are heterogeneous, data is scarce and skewed, and a node's strongest neighbors may far exceed its own local capacity. We study how nodes should train so that their predictions compose well at deployment, and how each node should learn whom to trust. Under a server-free, model-agnostic protocol where nodes exchange only queries and soft predictions, we propose Learned Neighbor Trust (LNTrust) wherein each node learns a compact trust function over its neighborhood from local validation evidence. This trust function gates auxiliary distillation during training and defines a deployment ensemble at inference, so that collaboration learned during training transfers directly to deployment. Across datasets and topologies, LNTrust improves deployed accuracy over the strongest output-only baseline by large margins while using significantly less communication than previous methods.
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2605.05009 [cs.LG] |
| (or arXiv:2605.05009v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2605.05009 arXiv-issued DOI via DataCite (pending registration) |
From: Michael Lanier [view email]
[v1]
Wed, 6 May 2026 15:07:40 UTC (2,599 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。