























Abstract:We study feed-forward ReLU networks with fixed readout and quadratic loss, and rewrite gradient descent as a collective dynamics of activation fields and conjugate fields on the training set. Working to first order in the learning rate inside a fixed activation chamber, we derive explicitly the one-, two- and three-hidden-layer cases, and then give the arbitrary-depth recursion. For one hidden layer the activation dynamics closes directly and the residual update is governed by the product of an input Gram matrix and a co-activation/backpropagation Gram matrix. For two hidden layers a conjugate field is required, but no nontrivial pullback Gram metric has yet appeared. For three hidden layers the first weight-induced pullback Gram metric enters the conjugate-field dynamics. At arbitrary depth, activation variations propagate forward through a recursive response operator \(\cU_\ell^{\alpha\beta}\), while conjugate-field variations propagate backward through an effective transport operator \(\cM_\ell^{\alpha\beta}\). Their contractions reconstruct a layerwise residual kernel \[ K_{\alpha\beta}^{(L)}=\sum_{\ell=1}^{L}Q_{\alpha\beta}^{(\ell-1)}S_{\alpha\beta}^{(\ell)}. \] The resulting description exposes a duality between push-forward and pullback transport across every layer cut, and identifies the first Gram metrics as the lowest nontrivial terms in a broader hierarchy of activation-conditioned transport operators. We deliberately stop at the level of collective fields, conjugate fields, residual kernels and cut-wise transport metrics, leaving the later tensorial geometric formulation outside the scope of this paper.
From: Claudio Nordio [view email]
[v1]
Mon, 8 Jun 2026 17:05:38 UTC (14 KB)
[v2]
Tue, 9 Jun 2026 11:03:48 UTC (16 KB)
[v3]
Wed, 10 Jun 2026 15:20:28 UTC (16 KB)
[v4]
Sat, 11 Jul 2026 09:49:15 UTC (10 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理,仅供阅读参考。 原文来自 — 版权归原作者所有。