Quality, Readability, and Content Deficiencies of Three Large Language Models in Bee Sting Health Information: A Comparative Evaluation with Patient Perspectives

Abstract Objective To evaluate the information quality, reliability, readability, and content deficits of ChatGPT, DeepSeek, and Doubao in answering frequently asked questions about bee stings. Methods Twenty-five high-frequency patient questions were selected based on Google Trends, domestic health platforms, and allergy guidelines, and submitted to the three LLMs, yielding 75 responses. Two emergency physicians independently assessed quality using GQS, mDISCERN, QAMAI, EQIP, and PEMAT-U/A. Readability was evaluated using six indices (ARI, FRES, GFI, FKGL, CLI, SMOG). Content deficits were classified qualitatively. Twenty emergency department patients evaluated the comprehensibility and credibility of AI responses. Results Significant differences were found among the three models on all quality metrics (all P Conclusion The three LLMs exhibited distinct strengths: DeepSeek in source reliability, ChatGPT in overall quality, and Doubao in understandability. All models had excessively high readability levels, and outdated medical advice was the predominant content deficit.