Abstract:Optimization algorithms are fundamental to modern deep learning, yet most widely used methods rely on update rules based primarily on local gradient statistics. We introduce NeuroPlastic, a plasticity-modulated optimizer that augments gradient-based updates with an adaptive multi-signal modulation mechanism inspired by multi-factor synaptic plasticity, a concept from neurobiology. NeuroPlastic dynamically scales gradient updates using interacting components that capture gradient, activity-like, and memory-like statistics, forming a lightweight modulation layer compatible with standard deep learning training pipelines. Across image classification benchmarks, NeuroPlastic consistently improves over a controlled gradient-only ablation, with more pronounced gains on the Fashion-MNIST benchmark and in reduced-data regimes. In transfer experiments on CIFAR-10 with ResNet-18, the method remains stable and competitive without retuning. These results suggest that multi-signal plasticity-inspired modulation can provide a useful extension to conventional gradient-driven optimization, particularly when learning signals are limited or noisy, and offer a promising direction for gradient-based methods in deep learning.
| Comments: | 16 pages, 7 figures |
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2604.26297 [cs.LG] |
| (or arXiv:2604.26297v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2604.26297 arXiv-issued DOI via DataCite (pending registration) |
From: Feng Tian [view email]
[v1]
Wed, 29 Apr 2026 04:52:21 UTC (2,305 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。