

















Authors:Junpeng Ding, Zichen Tang, Haihong E, Mengyuan Ji, Yang Liu, Haolin Tian, Haiyang Sun, Pengqi Sun, Yang Xu, Yichen Liu, Haocheng Gao, Zijie Xi, Ruomeng Jiang, Peizhi Zhao, Rongjin Li, Yuanze Li, Jiacheng Liu, Zhongjun Yang, Jintong Chen, Siying Lin
Abstract:We introduce SPUR, a comprehensive benchmark for scientific experimental image perception, understanding, and reasoning, comprising 4,264 question-answering (QA) pairs derived from 1,084 expert-curated images. SPUR features three key innovations: (1) Panel-Level Fine-Grained Perception: evaluating the visual perception of multimodal large language models (MLLMs) across three dimensions (numerical, morphological, and information localization) on six fine-grained panel types; (2) Cross-Panel Relation Understanding: utilizing complex images with an average of 14.3 panels per sample to evaluate MLLMs' ability to decipher intricate cross-panel relations; (3) Expert-Level Reasoning: assessment of qualitative and quantitative reasoning across five experimental paradigms to determine if models can infer conclusions from evidence as human experts do. Comprehensive evaluation of 20 MLLMs and four multimodal Chain-of-Thought (MCoT) methods reveals that current models fall significantly short of the expert-level requirements for scientific image interpretation, underscoring a critical bottleneck in AI for Science (AI4S) research.
| Comments: | Accepted to ACL 2026 Main Conference |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Computational Engineering, Finance, and Science (cs.CE) |
| Cite as: | arXiv:2604.27604 [cs.CV] |
| (or arXiv:2604.27604v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2604.27604 arXiv-issued DOI via DataCite |
From: Junpeng Ding [view email]
[v1]
Thu, 30 Apr 2026 08:57:18 UTC (27,257 KB)
[v2]
Tue, 26 May 2026 07:05:48 UTC (27,227 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理,仅供阅读参考。 原文来自 — 版权归原作者所有。