惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

G
Google Developers Blog
宝玉的分享
宝玉的分享
月光博客
月光博客
B
Blog
云风的 BLOG
云风的 BLOG
Google DeepMind News
Google DeepMind News
Engineering at Meta
Engineering at Meta
aimingoo的专栏
aimingoo的专栏
N
Netflix TechBlog - Medium
博客园_首页
GbyAI
GbyAI
人人都是产品经理
人人都是产品经理
A
About on SuperTechFans
Y
Y Combinator Blog
L
LangChain Blog
有赞技术团队
有赞技术团队
D
Docker
爱范儿
爱范儿
博客园 - 司徒正美
H
Hackread – Cybersecurity News, Data Breaches, AI and More
让小产品的独立变现更简单 - ezindie.com
让小产品的独立变现更简单 - ezindie.com
酷 壳 – CoolShell
酷 壳 – CoolShell
Microsoft Security Blog
Microsoft Security Blog
D
DataBreaches.Net

stat updates on arXiv.org

Simultaneous Monitoring of Shape and Surface Color via 4D Point Clouds: A Registration-free Approach A Refined Generalization Analysis for Extreme Multi-class Supervised Contrastive Representation Learning Ensemble Distributionally Robust Bayesian Optimisation The Proxy Presumption: From Semantic Embeddings to Valid Social Measures Modulated learning for private and distributed regression with just a single sample per client device Query-efficient model evaluation using cached responses Functional-prior-based approaches to Bayesian PDE-constrained inversion using physics-informed neural networks Optimal Experiments for Partial Causal Effect Identification Order-Agnostic Autoregressive Modelling with Missing Data Grokking or Glitching? How Low-Precision Drives Slingshot Loss Spikes Tuning Derivatives for Causal Fairness in Machine Learning Spherical Flows for Sampling Categorical Data Bayesian Rain Field Reconstruction using Commercial Microwave Links and Diffusion Model Priors GRALIS: A Unified Canonical Framework for Linear Attribution Methods via Riesz Representation Sharp Capacity Thresholds in Linear Associative Memory: From Winner-Take-All to Listwise Retrieval Unified Framework of Distributional Regret in Multi-Armed Bandits and Reinforcement Learning Jacobian-Velocity Bounds for Deployment Risk Under Covariate Drift Self-Attention as Transport: Limits of Symmetric Spectral Diagnostics Perturbation is All You Need for Extrapolating Language Models Adapt or Forget: Provable Tradeoffs Between Adam and SGD in Nonstationary Optimization Realizable Bayes-Consistency for General Metric Losses Graph Convolutional Support Vector Regression for Robust Spatiotemporal Forecasting of Urban Air Pollution Segmenting Human-LLM Co-authored Text via Change Point Detection Stochastic Schrödinger Diffusion Models for Pure-State Ensemble Generation Understanding Self-Supervised Learning via Latent Distribution Matching The Geometric Mechanics of Contrastive Representation Learning: Alignment Potentials, Entropic Dispersion, and Cross-modal Divergence Imbalanced Classification under Capacity Constraints On the Spectral Structure and Objective Equivalence of Orthogonal Multilabel Fisher Discriminants Partially Observed Structural Causal Models First-Order Efficiency for Probabilistic Value Estimation via A Statistical Viewpoint
Over-Alignment vs Over-Fitting: The Role of Feature Learn...
Taesun Yeom, Taehyeok Ha, Jaeho Lee · 2026-02-01 · via stat updates on arXiv.org

Feature learning strength (FLS), i.e., the inverse of the effective output scaling of a model, plays a critical role in shaping the optimization dynamics of neural nets. While its impact has been extensively studied under the asymptotic regimes -- both in training time and FLS -- existing theory offers limited insight into how FLS affects generalization in practical settings, such as when training is stopped upon reaching a target training risk. In this work, we investigate the impact of FLS on generalization in deep networks under such practical conditions. Through empirical studies, we first uncover the emergence of an $\textit{optimal FLS}$ -- neither too small nor too large -- that yields substantial generalization gains. This finding runs counter to the prevailing intuition that stronger feature learning universally improves generalization. To explain this phenomenon, we develop a theoretical analysis of gradient flow dynamics in two-layer ReLU nets trained with logistic loss, where FLS is controlled via initialization scale. Our main theoretical result establishes the existence of an optimal FLS arising from a trade-off between two competing effects: An excessively large FLS induces an $\textit{over-alignment}$ phenomenon that degrades generalization, while an overly small FLS leads to $\textit{over-fitting}$.