Abstract:Vision-Language Models (VLMs) are frequently undermined by object hallucination, generating content that contradicts visual reality, due to an over-reliance on linguistic priors. We introduce Positive-and-Negative Decoding (PND), a training-free inference framework that intervenes directly in the decoding process to enforce visual fidelity. PND is motivated by our finding of an attention imbalance in VLMs, where visual features are under-weighted. Our framework introduces a dual-path contrast: a positive path that amplifies visual evidence and a negative path that constructs counterfactuals to penalize prior-dominant generation. By contrasting outputs from both paths during decoding, PND steers generation toward visually grounded results. Experiments on POPE, MME, and CHAIR demonstrate state-of-the-art performance without retraining.
| Comments: | Accepted by CVPR 2026 (Conference on Computer Vision and Pattern Recognition). 11 pages, 5 figures. Code available at: this https URL |
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2605.06679 [cs.LG] |
| (or arXiv:2605.06679v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2605.06679 arXiv-issued DOI via DataCite |
From: Yubo Jiang [view email]
[v1]
Wed, 22 Apr 2026 15:11:41 UTC (26,198 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理，仅供阅读参考。 原文来自 — 版权归原作者所有。