Multi-task deep learning integrating pretreatment MRI and whole slide images predicts induction chemotherapy response and survival in locally advanced nasopharyngeal carcinoma

Predicting response to induction chemotherapy (IC) and overall survival (OS) is critical for optimizing treatment in patients with locally advanced nasopharyngeal carcinoma (LANPC). This study aimed to develop and validate a multi-task deep learning model integrating pretreatment MRI and whole slide images (WSIs) to predict IC response and OS in LANPC. Pretreatment MRI and WSIs from 404 patients with LANPC were retrospectively collected to construct a multi-task model (MoEMIL) for the simultaneous prediction of early IC response and OS. In our cohort, no progressive disease cases were observed; therefore, the poor responder group consisted exclusively of patients with stable disease. MoEMIL employed multi-instance learning to process WSIs, PyRadiomics and a convolutional neural network (ResNet50) to extract MRI features, and fused multimodal features through a multi-gate mixture-of-experts architecture. Clustering-constrained attention multiple instance learning and gradient-weighted class activation mapping were applied for visualization and interpretation. MoEMIL effectively stratified patients into good and poor IC response groups, achieving areas under the curve of 0.917, 0.869, and 0.801 in the train, validation, and test sets, respectively. In the test set, MoEMIL significantly outperformed the deep learning radiomics model and TNM staging for both endpoints. Compared to the Pathomics model, MoEMIL achieved significant improvement for OS ( P  = 0.0049) and borderline improvement for IC response ( P  = 0.064). The model could stratify patients into high- and low-risk OS groups ( P   0.05). MoEMIL might be an adjunctive tool for early induction chemotherapy response prediction and prognostic stratification in LANPC.