Abstract
Background
Mitotic-figure detectors are usually evaluated by detection accuracy, but use also depends on confidence and count reliability under domain shift.Methods
We evaluated MIDOG++ with leave-one-domain-out testing across seven domains. Adaptive-bin detection expected calibration error (D-ECE1) measured the bin-mass-weighted mean absolute confidence-precision gap among post-non-maximum suppression detections retained at a 0.05 score floor; centroid-distance matching supplied the true positive/false positive labels. F1 and count error were evaluated at the fixed 0.5 operating threshold. Baseline and domain-adversarial RetinaNet arms were compared as a controlled perturbation, with exploratory domain-composition resampling and two secondary seeds.Results
No directionally stable aggregate calibration advantage was established across three seed series. In the primary series, macro D-ECE1 was 0.0786 for baseline and 0.0741 for the domain-adversarial arm (paired difference, -0.0045), but two additional seed series produced paired differences of +0.0287 and -0.0068. A provenance audit confirmed the secondary series were generated under the locked protocol; they were reported as a sensitivity comparison and were not pooled with the primary point estimate. The fixed-domain case-bootstrap interval was below zero, whereas exploratory domain-composition resampling crossed zero (-0.0213 to +0.0112). The two domains with F1 gains had worse D-ECE1. Lymphoma/lymphosarcoma had the highest observed D-ECE1 in both arms and was systematically undercounted.Conclusions
In this benchmark, F1, emitted-detection calibration, count behavior, and maximum observed-domain D-ECE1 were not interchangeable endpoints. The tested domain-adversarial configuration did not establish a directionally stable calibration benefit. Evaluations should report detection-specific calibration, retained-detection denominators, count consequences, and finite-benchmark observed-domain D-ECE1 alongside F1.