Abstract:We propose Noise-Based Spectral Embedding (NBSE), a physics-informed framework for selecting informative features from high-dimensional data without greedy search. NBSE constructs a sparse similarity graph on the samples and identifies the Nishimori temperature $\beta_N$ the critical inverse temperature at which the Bethe Hessian becomes singular. The corresponding smallest eigenvector captures the dominant mode of an intrinsically degree-corrected diffusion process, naturally reweighting nodes to prevent hub dominance. By transposing the data matrix and applying NBSE in feature space, we obtain a one-dimensional spectral embedding that reveals groups of redundant or semantically related dimensions; balanced binning then selects one representative per group. We prove that coloured Gaussian perturbations shift $\beta_N$ by at most $O(\bar\sigma^2)$, guaranteeing robustness to measurement noise. Experiments on ImageNet embeddings from MobileNetV2 and EfficientNet-B4 show that NBSE preserves classification accuracy even under aggressive compression: on EfficientNet-B4 the accuracy drop is below $1\%$ when retaining only $30\%$ of features, outperforming ANOVA $F$-test and random selection by up to $6.8\%$.
| Comments: | 8 pages, 3 figures, extended version (with noise shift proof) of DSPA2026 article |
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2604.24692 [cs.LG] |
| (or arXiv:2604.24692v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2604.24692 arXiv-issued DOI via DataCite (pending registration) |
|
| Related DOI: | https://doi.org/10.1109/DSPA69176.2026.11476758
DOI(s) linking to related resources |
From: Vasiliy Usatyuk [view email]
[v1]
Mon, 27 Apr 2026 16:53:40 UTC (414 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。