




















Abstract:We identify a Selection Plateau phenomenon in one-shot neural network pruning: all rank-monotone weight scorers converge to identical accuracy at fixed sparsity, independent of functional form. We propose the Sparsity-Information-Complexity Spectrum (SICS) hypothesis: a sparsity-dependent minimum feature complexity kappa(S) governs plateau escape, with kappa=0 sufficient at low sparsity (S<0.65), kappa=1 dominant at critical sparsity (S~0.7), and kappa=2 necessary at extreme sparsity (S>0.75). On ViT-Small/CIFAR-10, testing nine feature classes across four sparsities, smooth non-monotone features provide +6.6% escape at S=0.7, while only raw features with high-frequency wiggle escape at S=0.8 (+2.6%). A fake non-monotone scorer underperforms the gradient baseline, indicating the requirement is magnitude-independent non-monotonicity. A handcrafted Gaussian bump achieves only +0.006 escape vs. chaos-derived +0.046, indicating rank-alignment is necessary but insufficient. SICS provides a unifying explanation for the performance clustering of diverse pruning methods and suggests that future selection algorithms should adapt feature complexity to target sparsity.
| Comments: | 22 pages, 3 figures, 5 tables. Empirical study + framework hypothesis on ViT-Small/CIFAR-10. Cross-domain validation (vision token pruning, KV cache compression, MoE routing) and cross-architecture extensions deferred to follow-up work |
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2605.09345 [cs.LG] |
| (or arXiv:2605.09345v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2605.09345 arXiv-issued DOI via DataCite (pending registration) |
From: Guangqi Li [view email]
[v1]
Sun, 10 May 2026 05:45:32 UTC (95 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理,仅供阅读参考。 原文来自 — 版权归原作者所有。