Abstract
Background:
The National Institutes of Health (NIH) recommends patient education materials (PEMs) be written below a seventh-grade reading level. Appropriate reading levels are important, as patients with mental health conditions and low health literacy are at greater risk of poor health outcomes.
Objective:
To assess and compare the readability level of NIH mental health PEMs using the Readable website and 2 artificial intelligence (AI) chatbots.
Methods:
Readability metrics (Flesch-Kinkaid reading level, Gunning Fog, Coleman-Liau, SMOG, and Automated Readability Index) of NIH mental health PEMs were evaluated using Readable, ChatGPT, and Claude AI in this cross-sectional descriptive study. Common readability issues were collected. Coloring pages and activity worksheets for children were excluded.
Results:
Forty PEMs were included in the final analysis. The average Flesch-Kincaid reading levels according to Readable, ChatGPT, and Claude were 8.74 (SD 1.85), 10.44 (SD 3.13), and 9.58 (SD 2.49), respectively. Average Gunning Fog, Coleman-Liau, SMOG, and Automated Readability Index scores provided by Readable were 10.51 (SD 2.09), 14.29 (SD 2.36), 11.27 (SD 1.46), and 10.18 (SD 2.13), respectively. The most common readability issue was sentences exceeding 20 syllables. Limitations included a single-time-point readability assessment of NIH mental health PEMs limited to 2 AI chatbots.
Conclusion:
NIH mental health PEMs consistently exceeded the seventh-grade reading level and should be revised to adhere to NIH recommendations. Readability metrics varied across Readable and AI chatbot platforms. Improving PEM readability enhances patient understanding, promotes informed decision-making, and reduces disparities linked to limited health literacy, particularly within mental health populations.