Abstract:Modern deep learning treats neural networks primarily as endpoint functions from inputs to outputs. Inspired by the shift from force to geometry in physics, we ask whether a network should instead be understood through the geometry of its internal propagation. We define a neural propagation field as the collection of hidden-state trajectories and local Jacobian operators across depth. Endpoint losses constrain only the boundary behavior of this field, leaving its interior geometry underdetermined. We show that endpoint-equivalent models can differ by orders of magnitude in trajectory and Jacobian structure, and introduce observable field metrics such as path sensitivity, solver consistency, and trajectory/Jacobian retention. In controlled teacher-flow and PDE systems, endpoint fitting fails to recover the underlying propagation law. In real multi-path tasks, field-aware objectives improve unseen-path generalization, OOD robustness, and calibration when aligned with the observation structure, but can collapse when over-constrained. In continual learning, field-preservation regularization complements replay and distillation: on Split CIFAR-100, DER++ with field preservation improves average accuracy, backward transfer, and field-retention metrics. These results identify propagation-field quality as a measurable and trainable property of neural networks beyond endpoint performance.
| Comments: | Technical notes on exploring the nature of deep learning propagation, Under review by the ICML 4th Workshop on High-dimensional Learning Dynamics (HiLD) 2026 |
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2605.08529 [cs.LG] |
| (or arXiv:2605.08529v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2605.08529 arXiv-issued DOI via DataCite (pending registration) |
|
| Journal reference: | ICML 2026 |
From: Xingrui Gu [view email]
[v1]
Fri, 8 May 2026 22:26:50 UTC (54 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。