Abstract
Cabled ocean observatories and coastal hydrophone stations accumulate large archives of ship-radiated noise, and underwater acoustic target recognition (UATR) systems are expected to work across such archives; yet their deep networks are trained and evaluated almost always within a single dataset, and cross-archive generalization remains unmeasured. We benchmark four ship-radiated-noise datasets—Oceanship from the Strait of Georgia, QiandaoEar22 from a freshwater lake, VTUAD rebuilt from public Ocean Networks Canada archives by AIS auto-labeling, and near-shore ShipsEar—under one preprocessing chain, frozen recording/clip-disjoint splits, and an ImageNet-pretrained ResNet-18 classifier. Within our mapping table's operationally comparable label space, direct transfer degrades severely in all twelve cross-dataset directions: accuracies 0.0–47.9% sit far below the 55.6–88.9% majority-class baselines of evaluation subsets. For the same-region Oceanship→VTUAD pair, the tuned checkpoint reaches only 0.9%, the sqrt-weighted checkpoint 2.9%, so dataset shift cannot be attributed to water body alone; label provenance, scenario construction, sensor configuration, vessel population, and temporal sampling remain plausible contributors. Centroid-level analysis shows cross-dataset gaps 5–17 times within-dataset inter-class distances. Under Oceanship's 4,278:1 imbalance, sqrt-smoothed class weights raise macro-F1 from 0.1302 to 0.1452, while inverse-frequency weighting destabilizes training. Two control architectures (a randomly initialized ResNet-18 and a small convolutional recurrent network) reproduce the same failure pattern in all twelve directions, and a ten-configuration ablation favors focal loss and macro-F1-based checkpoint selection (macro-F1 0.1812 ± 0.0033). Mapping table, frozen splits, and evaluation code are archived on Zenodo (latest version: 10.5281/zenodo.22906115; exact snapshot analyzed in this paper: v1.1, 10.5281/zenodo.23028094).