Performance of a HER2 testing algorithm tailored for urothelial bladder cancer: A Bi-centre study

Abstract Aims This study aimed to develop an AI algorithm for automated HER2 scoring in urothelial bladder cancer (UBCa) and assess the interobserver agreement using both manual and AI-assisted methods based on breast cancer criteria. Methods and Results We utilized 330 slides from two institutions for initial AI development and selected 200 slides for the ring study, involving six pathologists (3 senior, 3 junior). Our AI algorithm achieved high accuracy in two independent tests, with accuracies of 0.94 and 0.92. In the ring study, the AI-assisted method improved both accuracy (0.66 vs 0.94) and consistency (kappa=0.48; 95 % CI, 0.443–0.526 vs kappa=0.87; 95 % CI, 0.852–0.885) compared to manual scoring, especially in HER2-low cases (F1-scores: 0.63 vs 0.92). Additionally, in 62.3 % of heterogeneous HER2-positive cases, the interpretation accuracy significantly improved (0.49 vs 0.93). Pathologists, particularly junior ones, experienced enhanced accuracy and consistency with AI assistance. Conclusions This is the first study to provide a quantification algorithm for HER2 scoring in UBCa to assist pathologists in diagnosis. The ring study demonstrated that HER2 scoring based on breast cancer criteria can be effectively applied to UBCa. Furthermore, AI assistance significantly improves the accuracy and consistency of interpretations among pathologists with varying levels of experience, even in heterogeneous cases.