Matching Surgical Datasets to Prediction Tasks: NSQIP and Non-NSQIP Artificial Intelligence Models Across Specialties

Background: Surgical artificial intelligence models depend on training data that may not capture predictors required for specific outcomes. We compared discrimination of models derived from the American College of Surgeons National Surgical Quality Improvement Program (NSQIP) with models using non-NSQIP datasets across surgical specialties and prediction tasks. Study Design: Structured searches of PubMed, Scopus, and EMBASE identified adult surgical prediction models published from October 1, 2015, through January 31, 2026. The primary metric was area under the receiver-operating characteristic curve (AUROC). Multivariable ordinary least squares regression with study-level cluster-robust standard errors estimated adjusted differences in AUROC between data sources. Results: Seventy-five studies contributed 295 models: 145 NSQIP and 150 non-NSQIP. Mean AUROC was 0.71±0.09 for NSQIP vs 0.80±0.10 for non-NSQIP models (p Conclusions: Discrimination differences between NSQIP- and non-NSQIP-derived surgical prediction models were outcome-dependent and sensitive to precision weighting. These findings support matching training datasets to prediction tasks rather than assuming database size or standardization ensures superior discrimination.