Abstract:Multimodal learning has become a prominent research area, with the potential of substantial performance gains by combining information across modalities. At the same time, model development has trended toward increasingly complex deep learning architectures, motivated by the assumption that multimodal-specific methods improve performance. We challenge this assumption through a large-scale empirical study by reimplementing 19 high-impact multimodal methods across nine diverse datasets with up to 23 modalities. Under standardized experimental conditions, including hyperparameter tuning, weight initialization, cross-validation, and statistical testing, increased multimodal complexity often yields confusion rather than effective fusion of data modalities. Accordingly, complex multimodal architectures do not reliably outperform unimodal baselines and a Simple Baseline for Multimodal Learning (SimBaMM). Through a focused case study, we further demonstrate concrete methodological shortcomings even in top-tier multimodal learning publications, underscoring the need for standardized evaluation practices. In summary, we argue for a shift in focus for multimodal learning: away from the pursuit of architectural novelty and toward methodological rigor.
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2512.22991 [cs.LG] |
| (or arXiv:2512.22991v3 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2512.22991 arXiv-issued DOI via DataCite |
From: Tillmann Rheude [view email]
[v1]
Sun, 28 Dec 2025 16:20:36 UTC (256 KB)
[v2]
Wed, 14 Jan 2026 13:38:18 UTC (258 KB)
[v3]
Thu, 7 May 2026 08:56:54 UTC (279 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。