Abstract:Equilibrium Propagation (EP) is a biologically inspired local learning rule first proposed for convergent recurrent neural networks (CRNNs), in which synaptic updates depend only on neuron states from two distinct phases. EP estimates gradients that closely align with those computed by Backpropagation Through Time (BPTT) while significantly reducing computational demands, positioning it as a potential candidate for on-chip training in neuromorphic architectures. However, prior studies on EP have been constrained to shallow architectures, as deeper networks suffer from the vanishing gradient problem, leading to convergence difficulties in both energy minimization and gradient computation. To alleviate the vanishing gradient problem in deep EP networks, we propose a novel EP framework that incorporates layer-wise learning signals to provide auxiliary supervision, which enhances the convergence of neuron dynamics. This is the first work to integrate knowledge distillation and local error signals into EP, enabling the training of significantly deeper architectures. Our proposed approach achieves state-of-the-art performance on the CIFAR-10 and CIFAR-100 datasets, showcasing its scalability on deep VGG architectures. These results represent a significant advancement in the scalability of EP, suggesting that intermediate learning signals can extend the practical applicability of EP to deeper architectures.
| Subjects: | Machine Learning (cs.LG); Emerging Technologies (cs.ET) |
| Cite as: | arXiv:2508.15989 [cs.LG] |
| (or arXiv:2508.15989v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2508.15989 arXiv-issued DOI via DataCite |
From: Abhronil Sengupta [view email]
[v1]
Thu, 21 Aug 2025 22:19:30 UTC (439 KB)
[v2]
Fri, 8 May 2026 16:14:11 UTC (461 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。