惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

U
Unit 42
V
V2EX
Martin Fowler
Martin Fowler
博
博客园 - Franky
P
Proofpoint News Feed
P
Palo Alto Networks Blog
H
Hackread – Cybersecurity News, Data Breaches, AI and More
B
Blog
The Register - Security
The Register - Security
Latest news
Latest news
S
Security @ Cisco Blogs
Simon Willison's Weblog
Simon Willison's Weblog
Recorded Future
Recorded Future
大猫的无限游戏
大猫的无限游戏
M
Microsoft Research Blog - Microsoft Research
Scott Helme
Scott Helme
T
Tailwind CSS Blog
cs.CV updates on arXiv.org
cs.CV updates on arXiv.org
Cyber Security Advisories - MS-ISAC
Cyber Security Advisories - MS-ISAC
Application and Cybersecurity Blog
Application and Cybersecurity Blog
T
True Tiger Recordings
有赞技术团队
有赞技术团队
I
Intezer
Cisco Talos Blog
Cisco Talos Blog
Hacker News - Newest:
Hacker News - Newest: "LLM"
The GitHub Blog
The GitHub Blog
cs.AI updates on arXiv.org
cs.AI updates on arXiv.org
T
Tenable Blog
博
博客园 - 叶小钗
Hugging Face - Blog
Hugging Face - Blog
Hacker News: Ask HN
Hacker News: Ask HN
S
Security Archives - TechRepublic
F
Future of Privacy Forum
爱范儿
爱范儿
PCI Perspectives
PCI Perspectives
H
Help Net Security
让小产品的独立变现更简单 - ezindie.com
让小产品的独立变现更简单 - ezindie.com
T
The Blog of Author Tim Ferriss
MyScale Blog
MyScale Blog
N
Netflix TechBlog - Medium
罗
罗磊的独立博客
Apple Machine Learning Research
Apple Machine Learning Research
MongoDB | Blog
MongoDB | Blog
Security Latest
Security Latest
美
美团技术团队
博
博客园 - 三生石上(FineUI控件)
S
Schneier on Security
量
量子位
C
CERT Recently Published Vulnerability Notes
SecWiki News
SecWiki News

cs.LG updates on arXiv.org

Bandit Convex Optimization with Gradient Prediction Adaptivity Calibration, Uncertainty Communication, and Deployment Readiness in CKD Risk Prediction: A Framework Evaluation Study Temporal Contrastive Transformer for Financial Crime Detection: Self-Supervised Sequence Embeddings via Predictive Contrastive Coding DualOptim+: Bridging Shared and Decoupled Optimizer States for Better Machine Unlearning in Large Language Models Detecting Atypical Clients in Federated Learning via Representation-Level Divergence LABO: LLM-Accelerated Bayesian Optimization through Broad Exploration and Selective Experimentation Holomorphic Neural ODEs with Kolmogorov-Arnold Networks for Interpretable Discovery of Complex Dynamics Algebraic Machine Learning for Small-to-Medium Datasets Is Competitive against Strong Standard Baselines Provable Robustness against Backdoor Attacks via the Primal-Dual Perspective on Differential Privacy Support-aware offline policy selection for advertising marketplaces Ex-GraphRAG: Interpretable Evidence Routing for Graph-Augmented LLMs ECPO: Evidence-Coupled Policy Optimization for Evidence-Certified Candidate Ranking Quantitative coronary calcification analysis for prediction of myocardial ischemia using non-contrast CT calcium scoring Frequency-Domain Regularized Adversarial Alignment for Transferable Attacks against Closed-Source MLLMs Models Can Model, But Can't Bind: Structured Grounding in Text-to-Optimization Local Covariate Selection for Average Causal Effect Estimation without Pretreatment and Causal Sufficiency Assumptions Tailoring Teaching to Aptitude: Direction-Adaptive Self-Distillation for LLM Reasoning Position: The Time for Sampling Is Now! Charting a New Course for Bayesian Deep Learning Provable Joint Decontamination for Benchmarking Multiple Large Language Models SCI-Defense: Defending Manipulation Attacks from Generative Engine Optimization Can Breath Biomarkers Causally Influence Blood Glucose? Investigating VOC-Mediated Modulation in Diabetes One LR Doesn't Fit All: Heavy-Tail Guided Layerwise Learning Rates for LLMs Skill Weaving: Efficient LLM Improvement via Modular Skillpacks What are the Right Symmetries for Formal Theorem Proving? Noise Schedule Design for Diffusion Models: An Optimal Control Perspective $\textit{BlockFormer}$ : Transformer-based inference from interaction maps MMD-Balls as Credal Sets: A PAC-Bayesian Framework for Epistemic Uncertainty in Test-Time Adaptation Correcting Class Imbalance in Prior-Data Fitted Networks for Tabular Classification CASE-NET: Deep Spatio-Temporal Representation Learning via Causal Attention and Channel Recalibration for Multivariate Time Series Classification Beyond Single Slot: Joint Optimization for Multi-Slot Guaranteed Display Advertising Alike Parts: A Feature-Informed Approach to Local and Global Prototype Explanations AutoMCU: Feasibility-First MCU Neural Network Customization via LLM-based Multi-Agent Systems Evaluation of Pipelines for Data Integration into Knowledge Graphs MOSS: Self-Evolution through Source-Level Rewriting in Autonomous Agent Systems Graph neural network explanations reveal a topological signature of disease-associated hubs in biological networks Compiling Agentic Workflows into LLM Weights: Near-Frontier Quality at Two Orders of Magnitude Less Cost ARC-STAR: Auditable Post-Hoc Correction for PDE Foundation Models Discovering Entity-Conditioned Lag Heterogeneity: A Lag-Gated Neural Audit Framework for Panel Time Series PeakFocus: Bridging Peak Localization and Intensity Regression via a Unified Multi-Scale Framework for Electricity Load Forecasting Machine learning prediction of obstructive coronary artery disease using opportunistic coronary calcium and epicardial fat assessments from CT calcium scoring scans When Are Teacher Tokens Reliable? Position-Weighted On-Policy Self-Distillation for Reasoning One-Way Policy Optimization for Self-Evolving LLMs Thermodynamic Irreversibility of Training Algorithms Optimal Guarantees for Auditing Rényi Differentially Private Machine Learning Harnesses for Inference-Time Alignment over Execution Trajectories Equilibrium Propagation and Hamiltonian Inference in the Diffusive Fitzhugh-Nagumo Model OPPO: Bayesian Value Recursion for Token-Level Credit Assignment in LLM Reasoning On-Policy Consistency Training Improves LLM Safety with Minimal Capability Degradation Objective-Induced Bias and Search Dynamics in Multiobjective Unsupervised Feature Selection ConTact: Contact-First Antibody CDR Design via Explicit Interface Reasoning Beyond Scalar Objectives: Expert-Feedback-Driven Autonomous Experimentation for Scientific Discovery at the Nanoscale Aerodynamic force reconstruction using physics-informed Gaussian processes Dropout Universality: Scaling Laws and Optimal Scheduling at the Edge-of-Chaos Beyond Euclidean Proximity: Repairing Latent World Models with Horizon-Matched Trajectory Reachability Metrics How Many Different Outputs Can a Transformer Generate? PEARL: Unbiased Percentile Estimation via Contrastive Learning for Industrial-Scale Livestream Recommendation An Improved Adaptive PID Optimizer with Enhanced Convergence and Stability for Deep Learning Reinforced Graph of Thoughts: RL-Driven Adaptive Prompting for LLMs Cross-domain benchmarks reveal when coordinated AI agents improve scientific inference from partial evidence On the Sample Complexity of Discounted Reinforcement Learning with Optimized Certainty Equivalents How Sparsity Allocation Shapes Label-Free Post-Pruning Recoverability Explainable AI for Data-Driven Design of High-Dimensional Predictive Studies From Sequential Nodes to GPU Batches: Parallel Branch and Bound for Optimal $k$-Sparse GLMs LCGuard: Latent Communication Guard for Safe KV Sharing in Multi-Agent Systems MDM-Prime-v2: Binary Encoding and Index Shuffling Enable Scaling of Diffusion Language Models Predicting Performance of Symbolic and Prompt Programs with Examples Measuring Cross-Modal Synergy: A Benchmark for VLM Explainability AgForce Enables Antigen-conditioned Generative Antibody Design The Illusion of Reasoning: Exposing Evasive Data Contamination in LLMs via Zero-CoT Truncation Visibility nowcasting in South Korea: a machine learning approach to class imbalance and distribution shift Tabular foundation models for robust calibration of near-infrared chemical sensing data Scalable On-Policy Reinforcement Learning via Adaptive Batch Scaling CausalGuard: Conformal Inference under Graph Uncertainty TONIC: Token-Centric Semantic Communication for Task-Oriented Wireless Systems Engineering Hybrid Physics-Informed Neural Networks for Next-Generation Electricity Systems: A State-of-the-Art Review Protein Thoughts: Interpretable Reasoning with Tree of Thoughts and Embedding-Space Flow Matching for Protein-Protein Interaction Discovery Leveraging Self-Paced Curriculum Learning for Enhanced Modality Balance in Multimodal Conversational Emotion Recognition I-SAFE: Wasserstein Coherence Metrics for Structural Auditing of Scientific AI Models EmoTrack: Robust Depression Tracking from Counseling Transcripts across Session Regimes ChronoMedicalWorld: A Medical World Model for Learning Patient Trajectories from Longitudinal Care Data Symbolic Density Estimation for Discrete Distributions Prototype-Guided Classification Sub-Task Decoupling Framework: Enhancing Generalization and Interpretability for Multivariate Time Series stable-worldmodel: A Platform for Reproducible World Modeling Research and Evaluation Short-Term-to-Long-Term Memory Transfer for Knowledge Graphs under Partial Observability Reasoning through Verifiable Forecast Actions: Consistency-Grounded RL for Financial LLMs Three Costs of Amortizing Gaussian Process Inference with Neural Processes Dynamic Mixture of Latent Memories for Self-Evolving Agents Representation Gap: Explaining the Unreasonable Effectiveness of Neural Networks from a Geometric Perspective Embedding-Based Federated Learning with Runtime Governance for Iron Deficiency Prediction Memory-R2: Fair Credit Assignment for Long-Horizon Memory-Augmented LLM Agents Expectation Consistency Loss: Rethink Confidence Calibration under Covariate Shift When to Switch, Not Just What: Transition Quality Prediction in Clash Royale Manifold-Guided Attention Steering IKNO: Infinite-order Kernel Neural Operators TBP-mHC: full expressivity for manifold-constrained hyper connections through transportation polytopes The Attribution Impossibility: No Feature Ranking Is Faithful, Stable, and Complete Under Collinearity A Reproducible Log-Driven AutoML Framework for Interpretable Pipeline Optimization in Healthcare Risk Prediction Double descent for least-squares interpolation on contaminated data: A simulation study Toward Understanding Adversarial Distillation: Why Robust Teachers Fail Same Architecture, Different Capacity: Optimizer-Induced Spectral Scaling Laws
Little by Little: Continual Learning via Incremental Mixture of Rank-1 Associative Memory Experts
Haodong Lu, · 2026-05-23 · via cs.LG updates on arXiv.org

View PDF HTML (experimental)

Abstract:Continual learning (CL) with large pre-trained models aims to incrementally acquire knowledge without catastrophic forgetting. Existing LoRA-based Mixture-of-Experts (MoE) methods expand capacity by adding isolated new experts while freezing old ones, but still suffer from redundancy, interference, routing ambiguity, and consequent forgetting. We investigate the issues stemming from coarse-grained expert granularity. Coarse-grained experts (e.g., high-rank LoRA) encode low-specialty information, leading to expert duplication/interference and routing degradation/confusion as experts accumulate. In this work, we propose MoRAM (Mixture of Rank-1 Associative Memory). Grounded in the view that weight matrices act as linear associative memories, MoRAM achieves CL as gradual incrementing of reusable atomic rank-1 experts as memory. Each rank-1 adapter acts as a fine-grained MoE expert or an associative memory unit. By viewing rank-1 adapters as key-value memory pairs, we eliminate explicit MoE-LoRA routers with self-activation, where each memory atom evaluates its relevance via its intrinsic key. The inference process thus becomes a robust, content-addressable retrieval over the incrementally accumulated memory. Extensive experiments on CLIP and LLMs show that MoRAM significantly outperforms state-of-the-art methods, achieving a better plasticity-stability trade-off, stronger generalization, and reduced forgetting. Project Page: this https URL.
Comments: Accepted at ICML2026. Project page: this https URL
Subjects: Machine Learning (cs.LG)
Cite as: arXiv:2506.21035 [cs.LG]
  (or arXiv:2506.21035v5 [cs.LG] for this version)
  https://doi.org/10.48550/arXiv.2506.21035

arXiv-issued DOI via DataCite

Submission history

From: Haodong Lu [view email]
[v1] Thu, 26 Jun 2025 06:19:05 UTC (3,609 KB)
[v2] Thu, 9 Oct 2025 05:43:44 UTC (3,636 KB)
[v3] Fri, 6 Feb 2026 02:59:44 UTC (3,713 KB)
[v4] Wed, 11 Feb 2026 00:58:13 UTC (3,714 KB)
[v5] Thu, 21 May 2026 00:15:19 UTC (3,734 KB)

此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。