Common misconceptions about validation studies

Information bias is common in epidemiology and can substantially diminish the validity of study results. Validation studies, in which an investigator compares the accuracy of a measure with a gold standard measure, are an important way to understand and mitigate this bias. More attention is being paid to the importance of validation studies in recent years, yet they remain rare in epidemiologic research and, in our experience, they remain poorly understood. Many epidemiologists have not had any experience with validations studies, either in the classroom or in their work. We present an example of misclassification of a dichotomous exposure to elucidate some important misunderstandings about how to conduct validation studies to generate valid information. We demonstrate that careful attention to the design of validation studies is central to determining how the bias parameters (e.g. sensitivity and specificity or positive and negative predictive values) can be used in quantitative bias analyses to appropriately correct for misclassification. Whether sampling is done based on the true gold standard measure, the misclassified measure or at random will determine which parameters are valid and the precision of those estimates. Whether or not the validation is done stratified by other key variables (e.g. by the exposure) will also determine the validity of those estimates. We also present sample questions that can be used to teach these concepts. Increasing the presence of validation studies in the classroom could have a positive impact on their use and improve the validity of estimates of effect in epidemiologic research.