Abstract
Although the benefits of machine learning are undeniable in healthcare, explainability plays a vital role in improving transparency and understanding the most decisive and persuasive variables for prediction. The challenge is to identify explanations that make sense to the biomedical expert. This work proposes discernibility as a new approach to faithfully reflect human cognition, based on the user's perception of a relationship between explanations and data for a given variable. A total of 50 participants (19 biomedical experts and 31 data scientists) evaluated their perception of the discernibility of explanations from both synthetic and human-based datasets (National Health and Nutrition Examination Survey). The low inter-rater reliability of discernibility (Intraclass Correlation Coefficient < 0.5), with no significant difference between areas of expertise or levels of education, highlights the need for an objective metric of discernibility. Thirteen statistical coefficients were evaluated for their ability to capture, for a given variable, the relationship between its values and its explanations using Passing-Bablok regression. Among these, dcor was shown to be a reliable metric for assessing the discernibility of explanations, effectively capturing the clarity of the relationship between the data and their explanations, and providing clues to underlying pathophysiological mechanisms not immediately apparent when examining individual predictors. Discernibility can also serve as an evaluation metric for model quality, helping to prevent overfitting and aiding in feature selection, ultimately providing medical practitioners with more accurate and persuasive results.