Abstract
Background/Objectives: A model explanation can still look clinically credible after the model has stopped travelling well. Hospital transfer studies rarely test that mismatch directly. We examined whether predictions and feature attributions changed together when sepsis risk models moved between two hospital systems. Methods: We analyzed the public PhysioNet/Computing in Cardiology Challenge 2019 data. The cohort included adult stays with at least 24 hourly records and no positive sepsis label during the first 12 h. We derived 115 predictors from that 12-h window, then trained elastic-net logistic regression, random forest, and XGBoost models separately in systems A and B. Each fitted model was evaluated in its source system and, without target-site refitting, in the other system. We assessed calibration, a frozen source threshold chosen for 80% sensitivity, decision curves, subgroup performance, and 1000-resample bootstrap intervals. The XGBoost audit compared global SHAP rankings, top-10 overlap, attribution direction, missingness-attribution mass, an ablation without observation indicators, and classifiers trained to identify the hospital system. Results: The cohort comprised 30,646 stays with 1843 subsequent sepsis events. XGBoost AUROC fell from 0.725 internally to 0.608 after A→B transfer and from 0.702 to 0.636 after B→A transfer. External calibration slopes were 0.431 and 0.700. SHAP-rank correlations remained high at 0.949 and 0.917, yet the top-10 Jaccard overlap was only 0.667 in either direction; the within-site bootstrap reference was 1.000. Observation indicators contributed 24.7% of external attribution mass for A→B and 7.5% for B→A. On their own, those indicators identified the hospital system with AUROC 0.998, although removing them changed external AUROC by less than 0.01. Conclusions: The two hospitals left a clear signature in what clinicians measured and what they left unmeasured. That signature altered the explanation profile even when its incremental contribution to discrimination was small. External validation should test prediction, calibration, threshold behavior, and explanation stability together.