Abstract:Mixture-of-Experts (MoE) architectures have gained significant traction for reducing computational costs in deep neural networks by activating only a sparse subset of parameters during inference. While this efficiency makes MoE highly attractive for scaling vision tasks, its patch-based processing mechanism inherently disrupts traditional, routing-agnostic backdoor attacks by fragmenting or discarding adversarial triggers. To expose the vulnerabilities of this architecture, we introduce BadPatches, a novel routing-aware trigger application strategy specifically designed for patch-based MoE (pMoE) models and MoE-based vision transformers. Rather than applying a global pattern across the entire image, BadPatches encapsulates triggers within targeted image patches, ensuring they are consistently routed to and processed by the active experts. Our evaluations demonstrate that BadPatches achieves a high Attack Success Rate (ASR) at lower poisoning rates than routing-agnostic triggers, reaching over 83.2% ASR with a poisoning rate of only 0.01%, and scaling to a 96.8% ASR at 0.05%, while preserving the model's clean accuracy. Furthermore, the attack remains effective in gray-box scenarios where the adversary lacks complete knowledge of the model's patch routing configuration. Finally, we evaluate fine-pruning as a potential defense mechanism, revealing that pruning alone is insufficient to mitigate the attack; successful backdoor removal strictly requires the fine-tuning stage. These findings highlight the fragility of sparse vision architectures and underscore the need for routing-aware defenses.
From: Jona Te Lintelo [view email]
[v1]
Sat, 3 May 2025 12:48:04 UTC (615 KB)
[v2]
Fri, 15 Aug 2025 08:56:54 UTC (409 KB)
[v3]
Mon, 12 Jan 2026 08:42:28 UTC (193 KB)
[v4]
Sat, 29 Aug 2026 17:08:01 UTC (7,121 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。