Abstract
How do we validate tests of cognitive abilities? We argue that the validation process depends fundamentally on the nature of the construct under investigation. There are two the different traditions in the study of validity: one emphasizes a realist approach, asserting that variations in an underlying attribute must causally produce variation in measurement outcomes, while the other prioritizes the utility and predictive value of test scores. We distinguish between reflective constructs, which represent real entities with causal properties, and formative constructs, which serve as useful statistical summaries. We propose that seemingly conflicting conceptions of validity are not mutually exclusive but rather represent appropriate approaches for different types of constructs; reflective constructs warrant a causal-realist interpretation of validity, while a predictive-utilitarian approach is sufficient for the validation of formative constructs. Specifically, for reflective constructs, a reductionist approach is appropriate, and researchers should aim to ground their validation efforts in an understanding of the withinperson cognitive processes that underlie individual differences. Conversely, formative constructs are best validated through their predictive utility. By providing a typology of constructs and a guide for validation, this paper aims to help researchers apply the appropriate criteria for cognitive ability constructs.