

























Abstract:Empirical studies have shown that continuous low-loss paths can be constructed between independently trained neural network models. This phenomenon, known as mode connectivity, refers to the existence of such paths between distinct modes-i.e., well-trained solutions in parameter space. However, existing empirical methods do not reliably connect independently trained modes and have been evaluated mainly on a narrow set of architectures (e.g., basic CNNs, VGG, and ResNet), leaving their effectiveness on newer models unclear. In this work, we propose a new empirical algorithm for connecting independently trained modes that generalizes beyond traditional architectures and supports a broader range of networks, including MobileNet, ShuffleNet, EfficientNet, RegNet, Deep Layer Aggregation (DLA), and Compact Convolutional Transformers (CCT). In addition to broader applicability, the proposed method yields more consistent connectivity paths across independently trained mode pairs and supports connecting modes obtained with different training hyperparameters.
From: Yongding Tian [view email]
[v1]
Mon, 5 May 2025 12:16:55 UTC (4,618 KB)
[v2]
Tue, 10 Jun 2025 01:08:42 UTC (3,733 KB)
[v3]
Wed, 11 Jun 2025 01:28:36 UTC (2,012 KB)
[v4]
Mon, 22 Sep 2025 18:57:09 UTC (3,550 KB)
[v5]
Thu, 28 May 2026 10:26:24 UTC (5,675 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理,仅供阅读参考。 原文来自 — 版权归原作者所有。