Abstract
Standardized mean differences (SMDs) are used to express treatment effects on a common metric, yet there is no consensus on whether mean differences should be standardized using the outcome standard deviation (SD) at baseline or follow-up. This study examined whether this choice systematically affects effect-size magnitude in a reproducible sample of 100 randomized psychological clinical trials. For each trial, SMDs were calculated using either the pooled baseline SD (SMDpre) or the pooled follow-up SD (SMDpost). The unweighted paired mean difference showed that |SMDpre| was, 0.11 SMD units larger than |SMDpost|, 95% CI [0.07, 0.14]. In a multilevel meta-regression |SMDpre| was 0.082 SMD units larger than |SMDpost|, equalling a 13% increase, 95% CI [0.042, 0.123]. Bland–Altman analysis indicated substantial between-study variation, 95% limits of agreement [−0.25, 0.46]. Secondary analyses including 2,228 effect sizes across all follow-ups mirrored this pattern, indicating a 7% increase of SMD magnitude associated to |SMDpre|, 95% CI [0.021, 0.044]. Moderator analyses suggested that discrepancies were larger in studies applying symptom-threshold eligibility criteria, and in secondary analyses, using the same measure both to determine eligibility and to assess the outcome and with longer follow-up. Importantly, 70% of included studies did not report how SMDs were operationalized. These findings demonstrate that the choice of standardizing SD systematically affects estimated treatment-effect magnitudes and that the discrepancy can become more pronounced under specific design conditions. Researchers should therefore carefully consider the choice of standardizer and explicitly report the time point and method used to derive the SD.