Beyond Fully Supervised Pixel Annotations: Scribble-Drive...

Authors

Songlin Li School of Computer Science and Technology, Xinjiang University
Guofeng Yu School of Computer Science and Technology, Xinjiang University
Zhiqing Guo School of Computer Science and Technology, Xinjiang University Xinjiang Multimodal Intelligent Processing and Information Security Engineering Technology Research Center
Yunfeng Diao School of Computer Science and Information Engineering, Hefei University of Technology
Dan Ma School of Computer Science and Technology, Xinjiang University
Gaobo Yang College of Computer Science and Electronic Engineering, Hunan University

DOI:

https://doi.org/10.1609/aaai.v40i1.37033

Abstract

Deep learning-based image manipulation localization (IML) methods have achieved remarkable performance in recent years, but typically rely on large-scale pixel-level annotated datasets. To address the challenge of acquiring high-quality annotations, some recent weakly supervised methods utilize image-level labels to segment manipulated regions. However, the performance is still limited due to insufficient supervision signals. In this study, we explore a form of weak supervision that improves the annotation efficiency and detection performance, namely scribble annotation supervision. We re-annotated mainstream IML datasets with scribble labels and propose the first scribble-based IML (Sc-IML) dataset. Additionally, we propose the first scribble-based weakly supervised IML framework. Specifically, we employ self-supervised training with a structural consistency loss to encourage the model to produce consistent predictions under multi-scale and augmented inputs. In addition, we propose a prior-aware feature modulation module (PFMM) that adaptively integrates prior information from both manipulated and authentic regions for dynamic feature adjustment, further enhancing feature discriminability and prediction consistency in complex scenes. We also propose a gated adaptive fusion module (GAFM) that utilizes gating mechanisms to regulate information flow during feature fusion, guiding the model toward emphasizing potential tampered regions. Finally, we propose a confidence-aware entropy minimization loss. This loss dynamically regularizes predictions in weakly annotated or unlabeled regions based on model uncertainty, effectively suppressing unreliable predictions. Experimental results show that our method outperforms existing fully supervised approaches in terms of average performance both in-distribution and out-of-distribution.

How to Cite

Li, S., Yu, G., Guo, Z., Diao, Y., Ma, D., & Yang, G. (2026). Beyond Fully Supervised Pixel Annotations: Scribble-Driven Weakly-Supervised Framework for Image Manipulation Localization. Proceedings of the AAAI Conference on Artificial Intelligence, 40(1), 677–685. https://doi.org/10.1609/aaai.v40i1.37033

Issue

Section

AAAI Technical Track on Application Domains I