Abstract
Objectives
To compare the agreement, reliability and convergent validity of the five‐point ordinal Likert scale and a 100 mm visual analog scale (VAS) for three‐dimensional (3D) esthetic assessment of primary unilateral cleft lip repair using a counterbalanced crossover design.
Materials and Methods
Fifty lay observers (25 males, 25 females) evaluated 3D stereophotogrammetric facial models of 10 children with nonsyndromic unilateral cleft lip and palate at three time points: preoperative (T0), 1‐month postoperative (T1), and 6‐months postoperative (T2). Nine morphological items were assessed using both the five‐point Likert scale and a 100 mm VAS. A counterbalanced crossover design with a 2‐week washout period and inversion of scale order was implemented to minimize recall and order effects. Pairwise agreement analyses between the categorized VAS and the five‐point Likert scale (interscale, test–retest, presentation‐mode, and longitudinal agreement) were assessed using Cohen’s kappa (
κ
). Overall inter‐rater reliability was evaluated using the intraclass correlation coefficient (ICC), while convergent validity between the categorized VAS and the five‐point Likert scale was assessed using Spearman’s rank correlation coefficient.
Results
Pairwise agreement between the categorized VAS and the five‐point Likert scale ranged from
κ
= 0.54 to
κ
= 0.80, indicating moderate‐to‐substantial agreement. Overall inter‐rater reliability was excellent for both instruments (ICC = 0.849–0.982). Test–retest agreement was slight to fair (
κ
= −0.05 to 0.49), while agreement between full‐face and cropped image presentation was generally limited (
κ
= −0.01 to 0.43). Longitudinal agreement across surgical stages was uniformly slight (
κ
= −0.05 to 0.18), reflecting expected changes in postoperative morphology rather than measurement instability. Convergent validity between the categorized VAS and the five‐point Likert scale was excellent (Spearman’s
ρ
= 0.948–0.974; all
p
Conclusions
Both the five‐point Likert scale and the VAS demonstrated excellent overall inter‐rater reliability and excellent convergent validity. However, because agreement analyses required categorization of the continuous VAS, the present findings do not establish whether preservation of its continuous measurement properties would provide additional discriminatory capability.
Clinical Relevance
For 3D esthetic assessment of primary unilateral cleft lip repair, both the five‐point Likert scale and the VAS provide reliable group‐level evaluations. Standardized image presentation protocols and the use of multiple evaluators are recommended to improve the consistency of esthetic auditing.