Abstract
Abstract
Multimodal large language models (MLLMs) achieve strong performance in multimodal information extraction (MIE). In healthcare and elderly-care applications, restricted data access, privacy constraints, and scarce domain-specific annotations often favor inference with frozen models. Standard evaluation measures whether extracted results match reference labels but does not test whether a result changes when a text span identified by the model as its support is modified.
Controlled Text-Anchored Evidence-Dependence Verification for Low-Resource Multimodal Information Extraction (CEDV-MIE) implements this test as a bounded inference-time procedure. A text span reported as support for an initial schema-constrained extraction is modified under validity constraints, while the paired image, visual contextual records, and non-target text remain fixed. The original and perturbed extractions are compared to produce a prediction-level consistency record and a final schema-constrained extraction. No parameter updates or additional evidence annotations are required.
Across four public MIE benchmarks and four frozen backbones, CEDV-MIE improves average F₁ over direct prompting in all 16 dataset–backbone configurations, with a mean gain of 3.85 points (range: 1.48–10.78). Against a self-consistency control using seven chain-of-thought samples per input (SC-7), CEDV-MIE achieves higher average F₁ in 14 of 16 configurations under the same maximum allowance of seven backbone calls. The post-hoc Consistency Check Score (CCS) increases in all eight diagnostic configurations evaluated with two backbones (mean +0.195); evidence-stability accuracy and F₁ also increase on the validated diagnostic subset.