SA-GGCoT: social-aware graph-grounded chain-of-thought for multimodal claim verification under bounded evidence

Abstract Multimodal misinformation often reuses authentic images while shifting key contextual details such as time, location, identity, or event description, making verification harder than ordinary image-text matching. We study this problem under a bounded-evidence setting, where verification relies on limited external evidence and, when available, early social traces collected within a fixed post-publication window. We propose SA-GGCoT, a three-stage framework that uses checkpoint-aware state tracking to localize conflicting evidence, LLMs to convert selected evidence into structured rationales, and a heterogeneous evidence graph to verify evidence consistency. Experiments on Weibo, PHEME, and MR $$^2$$ 2 show consistent gains over protocol-matched baselines, with ablation and grounding analyses supporting the contribution of each component.