惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

V
V2EX
J
Java Code Geeks
MongoDB | Blog
MongoDB | Blog
Cyber Security Advisories - MS-ISAC
Cyber Security Advisories - MS-ISAC
Blog — PlanetScale
Blog — PlanetScale
A
About on SuperTechFans
Engineering at Meta
Engineering at Meta
美团技术团队
N
Netflix TechBlog - Medium
月光博客
月光博客
博客园 - 三生石上(FineUI控件)
F
Fortinet All Blogs
奇客Solidot–传递最新科技情报
奇客Solidot–传递最新科技情报
MyScale Blog
MyScale Blog
aimingoo的专栏
aimingoo的专栏
Last Week in AI
Last Week in AI
The Cloudflare Blog
量子位
Apple Machine Learning Research
Apple Machine Learning Research
小众软件
小众软件
腾讯CDC
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
D
DataBreaches.Net
H
Hackread – Cybersecurity News, Data Breaches, AI and More

stat updates on arXiv.org

A Refined Generalization Analysis for Extreme Multi-class Supervised Contrastive Representation Learning Ensemble Distributionally Robust Bayesian Optimisation The Proxy Presumption: From Semantic Embeddings to Valid Social Measures Modulated learning for private and distributed regression with just a single sample per client device Query-efficient model evaluation using cached responses Functional-prior-based approaches to Bayesian PDE-constrained inversion using physics-informed neural networks Optimal Experiments for Partial Causal Effect Identification Order-Agnostic Autoregressive Modelling with Missing Data Grokking or Glitching? How Low-Precision Drives Slingshot Loss Spikes Tuning Derivatives for Causal Fairness in Machine Learning Spherical Flows for Sampling Categorical Data Bayesian Rain Field Reconstruction using Commercial Microwave Links and Diffusion Model Priors GRALIS: A Unified Canonical Framework for Linear Attribution Methods via Riesz Representation Sharp Capacity Thresholds in Linear Associative Memory: From Winner-Take-All to Listwise Retrieval Unified Framework of Distributional Regret in Multi-Armed Bandits and Reinforcement Learning Jacobian-Velocity Bounds for Deployment Risk Under Covariate Drift Self-Attention as Transport: Limits of Symmetric Spectral Diagnostics Perturbation is All You Need for Extrapolating Language Models Adapt or Forget: Provable Tradeoffs Between Adam and SGD in Nonstationary Optimization Realizable Bayes-Consistency for General Metric Losses Graph Convolutional Support Vector Regression for Robust Spatiotemporal Forecasting of Urban Air Pollution Segmenting Human-LLM Co-authored Text via Change Point Detection Stochastic Schrödinger Diffusion Models for Pure-State Ensemble Generation Understanding Self-Supervised Learning via Latent Distribution Matching The Geometric Mechanics of Contrastive Representation Learning: Alignment Potentials, Entropic Dispersion, and Cross-modal Divergence Imbalanced Classification under Capacity Constraints On the Spectral Structure and Objective Equivalence of Orthogonal Multilabel Fisher Discriminants Partially Observed Structural Causal Models First-Order Efficiency for Probabilistic Value Estimation via A Statistical Viewpoint Robust and Fast Training via Per-Sample Clipping
Network Dynamics-Based Framework for Understanding Deep N...
Yuchen Lin, Yong Zhang, Sihan Feng, Hong Zhao · 2025-01-05 · via stat updates on arXiv.org

Advancements in artificial intelligence call for a deeper understanding of the fundamental mechanisms underlying deep learning. In this work, we propose a theoretical framework to analyze learning dynamics through the lens of dynamical systems theory. We redefine the notions of linearity and nonlinearity in neural networks by introducing two fundamental transformation units at the neuron level: order-preserving transformations and non-order-preserving transformations. Different transformation modes lead to distinct collective behaviors in weight vector organization, different modes of information extraction, and the emergence of qualitatively different learning phases. Transitions between these phases may occur during training, accounting for key phenomena such as grokking. To further characterize generalization and structural stability, we introduce the concept of attraction basins in both sample and weight spaces. The distribution of neurons with different transformation modes across layers, along with the structural characteristics of the two types of attraction basins, forms a set of core metrics for analyzing the performance of learning models. Hyperparameters such as depth, width, learning rate, and batch size act as control variables for fine-tuning these metrics. Our framework not only sheds light on the intrinsic advantages of deep learning, but also provides a novel perspective for optimizing network architectures and training strategies.