























Abstract:Predictive applications of machine learning often rely on small (sub 1 Bn parameter) specialized models tuned to particular domains or modalities. Such models often achieve excellent performance, but lack flexibility. LLMs and VLMs offer versatility, but typically underperform specialized predictors, especially on non-traditional modalities and long-tail domains. We propose MARVIS (Modality Adaptive Reasoning over VISualizations), a system that transforms latent embedding spaces into visual representations and then leverages the spatial and fine-grained reasoning skills of VLMs to interpret the visualizations and utilize them for predictions successfully. MARVIS achieves competitive performance across vision, audio, biological, and tabular domains using a single 3B parameter model, yielding results that beat Gemini 2.0 by 16% on average. MARVIS drastically reduces the gap between LLM/VLMs approaches and specialized domain-specific methods, without requiring any domain-specific training. Code and datasets are available at this https URL.
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2507.01544 [cs.LG] |
| (or arXiv:2507.01544v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2507.01544 arXiv-issued DOI via DataCite |
From: Benjamin Feuer [view email]
[v1]
Wed, 2 Jul 2025 09:56:24 UTC (37,627 KB)
[v2]
Wed, 29 Apr 2026 09:46:38 UTC (5,246 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理,仅供阅读参考。 原文来自 — 版权归原作者所有。