A new dataset and pipeline for ICAO compliance check of head coverings: an inclusive religious-aware approach

Abstract The automatic verification of the compliance of Machine-Readable Travel Documents’ (MRTD) portraits is nowadays of big interest for issuing authorities or application providers. Despite the increase of performance and effectiveness of automatic services, there is still much room for the improvement of such systems. Specific particularities related to cultural or religious aspects may increase inequalities or social tension between different people. Additionally, regulations and standards allow portraits a few differences related to these aspects, yet current automatic systems to verify their compliance generally do not take into account these particularities. This paper then contributes with a dataset and an algorithm that automatically verifies the compliance with the ICAO requirements related to the use of head coverings on facial images used on MRTDs. All the methods found in the literature ignore that some coverings might be accepted because of religious or cultural reasons, and mainly only look for the presence of hats or caps. Our approach specifically includes the religious cases and distinguishes the head coverings that might be considered compliant, depending on the national rules. We built a dataset composed of 6759 facial images of 910 identities to accommodate these types of head coverings. That data was used to fine-tune and train a new classification model derived from the YOLOv8 framework and we achieved state-of-the-art results with an overall accuracy on the created data of 91.05% and HTER of 8.53%.