A Hybrid Residual–Swin Transformer Design with Attention for Prostate Cancer Segmentation

Abstract Prostate cancer, a leading cause of cancer-related deaths among men globally, necessitates the development of precise diagnostic and treatment strategies. Accurate segmentation of prostate cancer in medical imaging, particularly in MRI scans, is crucial for early diagnosis and clinical decisions. Traditional manual segmentation techniques, while efficient, are labor-intensive and necessitate significant expertise, resulting in an increasing demand for automated alternatives. The RSAUNet is a novel deep learning architecture designed to improve prostate cancer segmentation. It incorporates essential components, including Residual Blocks, Swin Transformer Blocks, and Attention Mechanisms within a U-Net architecture. This markedly enhances the model’s capacity to discern complex anatomical features and accurately segment malignant tissues. Using sophisticated deep Learning methods, RSAUNet addresses the complexities of prostate imaging, delivering reliable, consistent segmentation results. The model was evaluated against various cutting-edge techniques on extensive multi-parametric MRI datasets, attaining an impressive Dice coefficient (DC) of 0.998 and a Jaccard Index (IoU) of 0.965. These findings highlight the innovative characteristics of RSAUNet and its capacity to transform prostate cancer diagnosis and treatment strategies. The proposed model surpasses current methods and shows potential for practical clinical applications, providing an efficient, precise, and scalable solution for automated prostate cancer segmentation and fostering optimism for the future of medical imaging and diagnosis.