惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

U
Unit 42
V
V2EX
Martin Fowler
Martin Fowler
博客园 - Franky
P
Proofpoint News Feed
P
Palo Alto Networks Blog
H
Hackread – Cybersecurity News, Data Breaches, AI and More
B
Blog
The Register - Security
The Register - Security
Latest news
Latest news
S
Security @ Cisco Blogs
Simon Willison's Weblog
Simon Willison's Weblog
Recorded Future
Recorded Future
大猫的无限游戏
大猫的无限游戏
M
Microsoft Research Blog - Microsoft Research
Scott Helme
Scott Helme
T
Tailwind CSS Blog
cs.CV updates on arXiv.org
cs.CV updates on arXiv.org
Cyber Security Advisories - MS-ISAC
Cyber Security Advisories - MS-ISAC
Application and Cybersecurity Blog
Application and Cybersecurity Blog
T
True Tiger Recordings
有赞技术团队
有赞技术团队
I
Intezer
Cisco Talos Blog
Cisco Talos Blog
Hacker News - Newest:
Hacker News - Newest: "LLM"
The GitHub Blog
The GitHub Blog
cs.AI updates on arXiv.org
cs.AI updates on arXiv.org
T
Tenable Blog
博客园 - 叶小钗
Hugging Face - Blog
Hugging Face - Blog
Hacker News: Ask HN
Hacker News: Ask HN
S
Security Archives - TechRepublic
F
Future of Privacy Forum
爱范儿
爱范儿
PCI Perspectives
PCI Perspectives
H
Help Net Security
让小产品的独立变现更简单 - ezindie.com
让小产品的独立变现更简单 - ezindie.com
T
The Blog of Author Tim Ferriss
MyScale Blog
MyScale Blog
N
Netflix TechBlog - Medium
罗磊的独立博客
Apple Machine Learning Research
Apple Machine Learning Research
MongoDB | Blog
MongoDB | Blog
Security Latest
Security Latest
美团技术团队
博客园 - 三生石上(FineUI控件)
S
Schneier on Security
量子位
C
CERT Recently Published Vulnerability Notes
SecWiki News
SecWiki News

cs.LG updates on arXiv.org

Models Can Model, But Can't Bind: Structured Grounding in Text-to-Optimization Dynamic Mixture of Latent Memories for Self-Evolving Agents On the Sample Complexity of Discounted Reinforcement Learning with Optimized Certainty Equivalents TONIC: Token-Centric Semantic Communication for Task-Oriented Wireless Systems MMD-Balls as Credal Sets: A PAC-Bayesian Framework for Epistemic Uncertainty in Test-Time Adaptation Alike Parts: A Feature-Informed Approach to Local and Global Prototype Explanations AutoMCU: Feasibility-First MCU Neural Network Customization via LLM-based Multi-Agent Systems ChronoMedicalWorld: A Medical World Model for Learning Patient Trajectories from Longitudinal Care Data The Illusion of Reasoning: Exposing Evasive Data Contamination in LLMs via Zero-CoT Truncation The Attribution Impossibility: No Feature Ranking Is Faithful, Stable, and Complete Under Collinearity Beyond Single Slot: Joint Optimization for Multi-Slot Guaranteed Display Advertising ConTact: Contact-First Antibody CDR Design via Explicit Interface Reasoning Three Costs of Amortizing Gaussian Process Inference with Neural Processes Symbolic Density Estimation for Discrete Distributions Machine learning prediction of obstructive coronary artery disease using opportunistic coronary calcium and epicardial fat assessments from CT calcium scoring scans Tabular foundation models for robust calibration of near-infrared chemical sensing data $\textit{BlockFormer}$ : Transformer-based inference from interaction maps On-Policy Consistency Training Improves LLM Safety with Minimal Capability Degradation Noise Schedule Design for Diffusion Models: An Optimal Control Perspective SCI-Defense: Defending Manipulation Attacks from Generative Engine Optimization Ex-GraphRAG: Interpretable Evidence Routing for Graph-Augmented LLMs Position: The Time for Sampling Is Now! Charting a New Course for Bayesian Deep Learning PEARL: Unbiased Percentile Estimation via Contrastive Learning for Industrial-Scale Livestream Recommendation Provable Joint Decontamination for Benchmarking Multiple Large Language Models When Are Teacher Tokens Reliable? Position-Weighted On-Policy Self-Distillation for Reasoning How Sparsity Allocation Shapes Label-Free Post-Pruning Recoverability DualOptim+: Bridging Shared and Decoupled Optimizer States for Better Machine Unlearning in Large Language Models A Reproducible Log-Driven AutoML Framework for Interpretable Pipeline Optimization in Healthcare Risk Prediction AgForce Enables Antigen-conditioned Generative Antibody Design Dropout Universality: Scaling Laws and Optimal Scheduling at the Edge-of-Chaos CausalGuard: Conformal Inference under Graph Uncertainty Quantitative coronary calcification analysis for prediction of myocardial ischemia using non-contrast CT calcium scoring Memory-R2: Fair Credit Assignment for Long-Horizon Memory-Augmented LLM Agents Beyond Scalar Objectives: Expert-Feedback-Driven Autonomous Experimentation for Scientific Discovery at the Nanoscale Equilibrium Propagation and Hamiltonian Inference in the Diffusive Fitzhugh-Nagumo Model I-SAFE: Wasserstein Coherence Metrics for Structural Auditing of Scientific AI Models Manifold-Guided Attention Steering When to Switch, Not Just What: Transition Quality Prediction in Clash Royale stable-worldmodel: A Platform for Reproducible World Modeling Research and Evaluation An Improved Adaptive PID Optimizer with Enhanced Convergence and Stability for Deep Learning Same Architecture, Different Capacity: Optimizer-Induced Spectral Scaling Laws LABO: LLM-Accelerated Bayesian Optimization through Broad Exploration and Selective Experimentation CASE-NET: Deep Spatio-Temporal Representation Learning via Causal Attention and Channel Recalibration for Multivariate Time Series Classification Temporal Contrastive Transformer for Financial Crime Detection: Self-Supervised Sequence Embeddings via Predictive Contrastive Coding Provable Robustness against Backdoor Attacks via the Primal-Dual Perspective on Differential Privacy Predicting Performance of Symbolic and Prompt Programs with Examples Discovering Entity-Conditioned Lag Heterogeneity: A Lag-Gated Neural Audit Framework for Panel Time Series TBP-mHC: full expressivity for manifold-constrained hyper connections through transportation polytopes Double descent for least-squares interpolation on contaminated data: A simulation study OPPO: Bayesian Value Recursion for Token-Level Credit Assignment in LLM Reasoning Leveraging Self-Paced Curriculum Learning for Enhanced Modality Balance in Multimodal Conversational Emotion Recognition Objective-Induced Bias and Search Dynamics in Multiobjective Unsupervised Feature Selection Expectation Consistency Loss: Rethink Confidence Calibration under Covariate Shift Embedding-Based Federated Learning with Runtime Governance for Iron Deficiency Prediction PeakFocus: Bridging Peak Localization and Intensity Regression via a Unified Multi-Scale Framework for Electricity Load Forecasting Representation Gap: Explaining the Unreasonable Effectiveness of Neural Networks from a Geometric Perspective Optimal Guarantees for Auditing Rényi Differentially Private Machine Learning Reasoning through Verifiable Forecast Actions: Consistency-Grounded RL for Financial LLMs Calibration, Uncertainty Communication, and Deployment Readiness in CKD Risk Prediction: A Framework Evaluation Study Harnesses for Inference-Time Alignment over Execution Trajectories Correcting Class Imbalance in Prior-Data Fitted Networks for Tabular Classification Hierarchical Variational Policies for Reward-Guided Diffusion Teaching Language Models to Forecast Research Success Through Comparative Idea Evaluation HealthCraft: A Reinforcement Learning Safety Environment for Emergency Medicine Check Your LLM's Secret Dictionary! Five Lines of Code Reveal What Your LLM Learned (Including What It Shouldn't Have) X-Token: Projection-Guided Cross-Tokenizer Knowledge Distillation EntmaxKV: Support-Aware Decoding for Entmax Attention From Parameters to Data: A Task-Parameter-Guided Fine-Tuning Pipeline for Efficient LLM Alignment Amplifying, Not Learning: Fine-Tuned AI Text Detectors Amplify a Pretrained Direction Value-Gradient Hypothesis of RL for LLMs Don't Collapse Your Features: Why CenterLoss Hurts OOD Detection and Multi-Scale Mahalanobis Wins Geometry-Adaptive Explainer for Faithful Dictionary-Based Interpretability under Distribution Shift Why Semantic Entropy Fails: Geometry-Aware and Calibrated Uncertainty for Policy Optimization Energy-Gated Attention: Spectral Salience as an Inductive Bias for Transformer Attention Memory-Efficient Partitioned DNN Inference on Resource-Constrained Android Crowds ReversedQ: Opportunities for Faster Q-Learning in Episodic Online Reinforcement Learning PACD-Net: Pseudo-Augmented Contrastive Distillation for Glycemic Control Estimation from SMBG Mechanistic Interpretability for Learning Assurance of a Vision-Based Landing System Compositional Transduction with Latent Analogies for Offline Goal-Conditioned Reinforcement Learning LT2: Linear-Time Looped Transformers The Devil is in the Condition Numbers: Why is GLU Better than non-GLU Structure? Learning to Think in Physics: Breaking Shortcut Learning in Scientific Diffusion via Representation Alignment The Hidden Signal of Verifier Strictness: Controlling and Improving Step-Wise Verification via Selective Latent Steering Distribution-Aware Reward: Reinforcement Learning over Predictive Distributions for LLM Regression Unsupervised clustering and classification of upper limb EMG signals during functional movements: a data-driven REFLECTOR: Internalizing Step-wise Reflection against Indirect Jailbreak The General Theory of Localization Methods Robust Recommendation from Noisy Implicit Feedback: A GMM-Weighted Bayes-label Transition Matrix Framework Decision-Path Patterns as Tree Reliability Signals: Path-based Adaptive Weighting for Random Forest Classification AGPO: Adaptive Group Policy Optimization with Dual Statistical Feedback ShapeBench: A Scalable Benchmark and Diagnostic Suite for Standardized Evaluation in Aerodynamic Shape Optimization Correcting Stochastic Update Bias in Preconditioned Language Model Optimizers Cumulative Meta-Learning from Active Learning Queries for Robustness to Spurious Correlations Dynamic Shapley Computation Same Target, Different Basins: Hard vs. Soft Labels for Annotator Distributions Design for Manufacturing: A Manufacturability Knowledge-Integrated Reinforcement Learning Framework for Free-Form Pipe Routing in Aeroengines Distributed Direct Preference Optimization Dynamic TMoE: A Drift-Aware Dynamic Mixture of Experts Framework for Non-Stationary Time Series Forecasting AVSD: Adaptive-View Self-Distillation by Balancing Consensus and Teacher-Specific Privileged Signals Hack-Verifiable Environments: Towards Evaluating Reward Hacking at Scale
Toward Understanding Adversarial Distillation: Why Robust Teachers Fail
Hongsin Lee, · 2026-05-23 · via cs.LG updates on arXiv.org

View PDF HTML (experimental)

Abstract:Adversarial Distillation aims to enhance student robustness by guiding the student with a robust teacher's soft labels within the min-max adversarial training framework, yet its success is notoriously inconsistent: a more robust teacher often fails to improve, or even harms, the student's robust generalization. In this paper, we identify a key mechanism of this teacher dependency: the misalignment between the teacher's supervisory confidence and the student's representational limitations on a consistent subset of training data -- the Robustly Unlearnable Set. We present a theoretical framework analyzing the feature learning dynamics of a two-layer neural network, demonstrating that this mismatch creates a dichotomy in distillation outcomes. We prove that when a teacher provides confident supervision on unlearnable samples, it compels the student to memorize spurious noise patterns that eventually overpower the learned robust signal, thereby driving robust overfitting. Conversely, a teacher that exhibits high uncertainty on these samples effectively suppresses noise memorization, allowing the student to rely solely on the learnable signal for robust generalization. We empirically validate our theory across both synthetic simulations and real-image classification datasets, confirming that robust overfitting is driven by the teacher's interaction with unlearnable samples. Finally, we demonstrate that a teacher's predictive entropy on unlearnable samples serves as a strong indicator of student robustness, validating our theoretical framework and offering a principled guideline for robust teacher selection.
Comments: Accepted to ICML 2026. Code is available at this https URL
Subjects: Machine Learning (cs.LG)
Cite as: arXiv:2605.21999 [cs.LG]
  (or arXiv:2605.21999v1 [cs.LG] for this version)
  https://doi.org/10.48550/arXiv.2605.21999

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Hongsin Lee [view email]
[v1] Thu, 21 May 2026 04:57:15 UTC (488 KB)