惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

Engineering at Meta
Engineering at Meta
J
Java Code Geeks
I
InfoQ
腾讯CDC
Vercel News
Vercel News
IT之家
IT之家
V
Visual Studio Blog
P
Proofpoint News Feed
freeCodeCamp Programming Tutorials: Python, JavaScript, Git & More
G
Google Developers Blog
Cyber Security Advisories - MS-ISAC
Cyber Security Advisories - MS-ISAC
博客园 - 叶小钗
有赞技术团队
有赞技术团队
月光博客
月光博客
Martin Fowler
Martin Fowler
量子位
L
LangChain Blog
B
Blog
Last Week in AI
Last Week in AI
博客园 - 司徒正美
Microsoft Security Blog
Microsoft Security Blog
博客园 - 聂微东
Microsoft Azure Blog
Microsoft Azure Blog
A
About on SuperTechFans

cs.CV updates on arXiv.org

ClawGUI: A Unified Framework for Training, Evaluating, and Deploying GUI Agents On the Robustness of Watermarking for Autoregressive Image Generation Revisiting Compositionality in Dual-Encoder Vision-Language Models: The Role of Inference Anthropogenic Regional Adaptation in Multimodal Vision-Language Model From Redaction to Restoration: Deep Learning for Medical Image Anonymization and Reconstruction The Salami Slicing Threat: Exploiting Cumulative Risks in LLM Systems BoxTuning: Directly Injecting the Object Box for Multimodal Model Fine-Tuning Semantic-Geometric Dual Compression: Training-Free Visual Token Reduction for Ultra-High-Resolution Remote Sensing Understanding Lightweight Low-Light Image Enhancement via Distribution-Normalizing Preprocessing and Depthwise U-Net Back to the Barn with LLAMAs: Evolving Pretrained LLM Backbones in Finetuning Vision Language Models Pseudo-Unification: Entropy Probing Reveals Divergent Information Patterns in Unified Multimodal Models QShield: Securing Neural Networks Against Adversarial Attacks using Quantum Circuits Evaluating the Impact of Medical Image Reconstruction on Downstream AI Fairness and Performance Retinal Cyst Detection from Optical Coherence Tomography Images LoViF 2026 The First Challenge on Weather Removal in Videos STORM: End-to-End Referring Multi-Object Tracking in Videos Data-Efficient Surgical Phase Segmentation in Small-Incision Cataract Surgery: A Controlled Study of Vision Foundation Models Rethinking the Diffusion Model from a Langevin Perspective Zero-shot World Models Are Developmentally Efficient Learners Edu-MMBias: A Three-Tier Multimodal Benchmark for Auditing Social Bias in Vision-Language Models under Educational Contexts VGA-Bench: A Unified Benchmark and Multi-Model Framework for Video Aesthetics and Generation Quality Evaluation Degradation-Consistent Paired Training for Robust AI-Generated Image Detection FREE-Switch: Frequency-based Dynamic LoRA Switch for Style Transfer Demographic and Linguistic Bias Evaluation in Omnimodal Language Models FlowPalm: Optical Flow Driven Non-Rigid Deformation for Geometrically Diverse Palmprint Generation Cross-Cultural Value Awareness in Large Vision-Language Models I Walk the Line: Examining the Role of Gestalt Continuity in Object Binding for Vision Transformers GLEaN: A Text-to-image Bias Detection Approach for Public Comprehension From UAV Imagery to Agronomic Reasoning: A Multimodal LLM Benchmark for Plant Phenotyping Not Your Stereo-Typical Estimator: Combining Vision and Language for Volume Perception
Bridging visual saliency and large language models for ex...
Paul Valery Nguezet, Elie Tagne Fute, Yusuf Brima, Benoit Martin · 2026-05-07 · via cs.CV updates on arXiv.org

The opaque nature of deep learning models remains a significant barrier to their clinical adoption in medical imaging. This paper presents a multimodal explainability framework that bridges the gap between convolutional neural network (CNN) predictions and clinically actionable insights for brain tumor classification, leveraging large language models (LLMs) to deliver human-interpretable diagnostic narratives. The proposed framework operates through three coupled stages. First, nine CNN architectures are extended with a dual-output hybrid formulation that simultaneously optimises a classification head and a segmentation head, enabling spatially richer feature learning. Second, visual saliency attribution methods, namely Grad-CAM, Grad-CAM++, and ScoreCAM, are applied to generate class-discriminative heatmaps, which are subsequently refined into binary tumor masks via an adaptive percentile thresholding pipeline. Third, the resulting masks are mapped onto the Harvard-Oxford cortical atlas to translate pixel-level evidence into named neuroanatomical structures, and the extracted findings are encoded into a structured JSON file that conditions three LLMs (Grok3, Mistral, and LLaMA) to generate coherent, radiological-style diagnostic reports. Evaluated on a dataset of 4,834 contrast-enhanced T1-weighted brain MRI images spanning three tumor classes, InceptionResNetV2 achieved the highest classification performance and Grad-CAM++ yielded the best segmentation overlap. Among the language models, Grok3 led in lexical diversity and coherence, while LLaMA achieved the highest readability score. By integrating visual, anatomical, and linguistic modalities into a unified pipeline, the framework produces explanations that are technically grounded and meaningfully interpretable, advancing the transparency and clinical accountability of artificial intelligence assisted brain tumor diagnosis.