Abstract
Abstract
Background
The COVID-19 pandemic has increased the demand for telepsychiatry in neurodevelopmental assessment; however, few studies have examined the interrater reliability of clinician-rated autism scales, such as the Childhood Autism Rating Scale, Second Edition (CARS2), when used remotely. Although the feasibility and validity of telehealth assessments have been explored previously, the consistency between remote and in person CARS2 ratings by different clinicians remains underinvestigated.
Objective
The objective of this study was to evaluate agreement between CARS2 total scores obtained by different evaluators using telepsychiatry and face-to-face assessments in children with autism spectrum disorder (ASD) and/or attention-deficit/hyperactivity disorder (ADHD).
Methods
In this randomized feasibility study, 75 children aged 6 to 17 years with
DSM-5 (Diagnostic and Statistical Manual of Mental Disorders [Fifth Edition])
diagnoses of ASD, ADHD, or both were enrolled at 5 medical institutions in Japan; 73 of 75 were included in the final analysis. Each participant underwent 2 independent CARS2 evaluations, one face-to-face and the other via telepsychiatry, conducted by different evaluators who were blinded to each other’s ratings. Remote assessments were conducted using a secure video platform with caregiver facilitation. Agreement was assessed using intraclass correlation coefficients (ICCs) for the overall sample and subgroups defined by primary diagnosis, sex, and age.
Results
The overall ICC for CARS2 total scores between face-to-face and telepsychiatry assessments was 0.582 (95% CI 0.407‐0.717), indicating moderate agreement. The ICC was 0.546 among participants with ASD as the primary diagnosis and 0.495 among those with ADHD as the primary diagnosis. Point estimates were higher among girls than boys (0.666 vs 0.495) and among children aged 11 years or older than among those aged 10 years or younger (0.676 vs 0.419). In a sensitivity analysis including all participants with an ASD diagnosis, the ICC was 0.580.
Conclusions
CARS2 total scores obtained through telepsychiatry showed moderate agreement with scores obtained through face-to-face assessment when the assessments were conducted by different evaluators in different settings. Higher point estimates were observed among older children and girls, although these subgroup findings should be interpreted cautiously. Telepsychiatry-based CARS2 assessment may be considered a complementary approach rather than a replacement for comprehensive face-to-face evaluation. Larger studies using more standardized assessment protocols are needed.