Heterogeneity in Meta-Analytic Effect Sizes: An Assessment of the Current State of the Literature

Heterogeneity refers to the variability in effect sizes across different samples and is one of the major criteria to judge the importance and advancement of a scientific area. To determine how studies in the organizational sciences address heterogeneity, we conduct two studies. In study 1, we examine how meta-analytic studies conduct heterogeneity assessments and report and interpret the obtained results. To do so, we coded heterogeneity-related information from meta-analytic studies published in five leading journals. We found that most meta-analytic studies report several heterogeneity statistics. At the same time, however, there tends to be a lack of detail and thoroughness in the interpretation of these statistics. In study 2, we review how primary studies report heterogeneity-related results and conclusions from meta-analyses. We found that the quality of the reporting of heterogeneity-related information in primary studies tends to be poor and unrelated to the detail and thoroughness with which meta-analytic studies report and interpret the statistics. Based on our findings, we discuss implications for practice and provide recommendations for how heterogeneity assessments should be conducted and communicated in future research.

Automated summary

The study finds that most meta-analytic studies report heterogeneity statistics, but lack detail and thoroughness in interpreting these statistics. Additionally, the quality of reporting heterogeneity-related information in primary studies is poor and unrelated to the detail and thoroughness of meta-analytic studies. Based on the findings, implications for practice and recommendations for future research on how to conduct and communicate heterogeneity assessments are discussed.