惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

S
SegmentFault 最新的问题
博客园 - 三生石上(FineUI控件)
爱范儿
爱范儿
博客园 - 聂微东
V
Visual Studio Blog
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
M
MIT News - Artificial intelligence
The GitHub Blog
The GitHub Blog
Recent Announcements
Recent Announcements
有赞技术团队
有赞技术团队
L
LangChain Blog
I
InfoQ
T
Tailwind CSS Blog
博客园 - 【当耐特】
V
V2EX
博客园_首页
奇客Solidot–传递最新科技情报
奇客Solidot–传递最新科技情报
GbyAI
GbyAI
Vercel News
Vercel News
雷峰网
雷峰网
量子位
A
About on SuperTechFans
Martin Fowler
Martin Fowler
H
Help Net Security

math updates on arXiv.org

Coupling-Robust Accuracy in Multiphysics Physics Informed Neural Networks via Kronecker-Preconditioned Optimization Non-normal spectral signatures of instability in neural network training dynamics Optimization of randomized neural networks for transfer operator approximation Selective Ambulance Dispatch Under Contextual Travel-Time Uncertainty LLAMA LIMA: A Living Meta-Analysis on the Effects of Generative AI on Learning Mathematics Learning Decision-Sufficient Representations for Linear Optimization Neural Flow Operators can Approximate any Operator: Abstract Frameworks and Universal Approximations LLMs as Noisy Channels: A Shannon Perspective on Model Capacity and Scaling Laws On the Stability of Spherical Hellinger-Kantorovich Flows and Their Implications for Differential Privacy Training-Free Looped Transformers Move on Muon : A Hamiltonian probability gradient flow perspective of Muon optimizer Entrywise Error Bounds for Spectral Ranking with Semi-Random Adversaries Asymmetric Scaling Laws from Sparse Features Is Dimensionality a Barrier for Retrieval Models? RA-DCA: A Randomized Active-Set DCA for Directional Stationarity in Max-Structured DC Programs Commutator-Induced Uncertainty in VAEs Weisfeiler-Leman Is Incomplete on Simple Spectrum Graphs, so Canonicalize Them Sparse In-Network Learning via Shortest-Path Backpropagation and Finite-Rate Gating Instance-Optimal Estimation with Multiple LLM Judges on a Budget Entropy Equivalence Testing Expand More, Shrink Less: Shaping Effective-Rank Dynamics for Dense Scaling in Recommendation Any-Dimensional Invariant Universality Operationalizing Individual Fairness via Gradient Descent and Bradley-Terry Models Anytime Training with Schedule-Free Spectral Optimization Diffusion-based Denoising Beats Vanilla Score Matching in Parameter Estimation: A Theoretical Explanation Resilience Characterization of AI-Native Wireless Receivers via Persistent Homology The General Theory of Localization Methods Group-Algebraic Tensors: Provably-optimal Equivariant Learning and Physical Symmetry Discovery General Lower Bounds for Differentially Private Federated Learning with Arbitrary Public-Transcript Interactions PilotWiMAE: Pilot-Native Representation Learning for Wireless Channels
Feature Learning Dynamics in Infinite-Depth Neural Networks
Zihan Yao, Ruoyu Wu, Tianxiang Gao · 2025-12-24 · via math updates on arXiv.org

Deep neural networks have achieved remarkable success in practice, yet a mechanistic understanding of how features evolve during training remains incomplete, especially in the large-depth limit. For ResNets under depth-$μ$P scaling, prior work treats the layer index $\ell$ as a continuous time $t_\ell = \ell/L$, yielding SDE descriptions of the training dynamics. A key unresolved issue is that backpropagation reuses each forward weight matrix $W_\ell$ through its transpose $W_\ell^\top$, creating correlations between forward features and backward gradients whose behavior and role in feature learning remain unclear. We study this reused-weight forward--backward coupling in one-layer ResNets under depth-$μ$P. Using conditional Gaussian representations, we explicitly separate the coupling terms induced by weight reuse from decoupled Gaussian fluctuations before taking any network limit. At initialization, we prove that the coupling is a finite-width effect and vanishes at rate $O(n^{-1})$, uniformly over depth. During training, however, SGD induces a nontrivial forward--backward correlation term that survives the infinite-width limit. The key depth effect is that, under depth-$μ$P scaling, this surviving term is higher order in depth and its accumulated contribution over layers becomes negligible as $L\to\infty$. This depth-induced suppression motivates Neural Feature Dynamics (NFD), a forward--backward SDE system with decoupled backward weights that retains the feature-gradient covariance structure generated during training. Under nondegeneracy assumptions, we prove that the finite-network training dynamics converge to its NFD limit with an $O(L^{-1})$ depth-discretization error, while the reused-weight coupling term has a faster $O(L^{-2})$ decay. These results provide a rigorous infinite-depth limit for the feature-learning dynamics of one-layer ResNets under depth-$μ$P.