Towards Robust Automated Mouse Scratching Behavior Detection for Itch Research

Abstract Automated detection of mouse scratching behavior could reduce manual annotation and is essential for preclinical itch research. However, robust performance across laboratories remains a challenge for current methods. We present Scratch-AID+, a framework dedicated to automated mouse scratching detection, including a diverse and progressive set of approaches: classic ResNet-based detectors, task-adaptive fine-tuning of video foundation models, and in-context learning with advanced multimodal large language models. To simulate real-world use cases, we benchmark on a public dataset and utilize additional cross-laboratory video recordings for independent testing. We find that classic baselines suffered severe performance degradation on external data under the realistic setting with sparse scratching events. In contrast, fine-tuned video foundation models successfully bridged this generalizability gap, achieving significantly better detection performance. Furthermore, frozen multimodal large language models demonstrated promising out-of-the-box ability using only few-shot demonstrations. We also examined the effects of different video preprocessing strategies. Our findings highlight the improved performance and generalizability of the adapted foundation models for automated scratching behavior analysis and shed light on the real-world deployment of such detectors. Our code is publicly available at https://anonymous.4open.science/r/scratch-aid-plus/ .