4.2 Article

Power Approximations for Overall Average Effects in Meta-Analysis With Dependent Effect Sizes

Journal

Publisher

SAGE PUBLICATIONS INC
DOI: 10.3102/10769986221127379

Keywords

power; meta-analysis; dependent effect sizes; robust variance estimation

Ask authors/readers for more resources

Meta-analytic models for dependent effect sizes have become increasingly complex, posing challenges for a priori power calculations. This study introduces power approximations for tests of average effect sizes based on common approaches for handling dependent effect sizes. Through Monte Carlo simulation, the study demonstrates that the new power formulas can accurately estimate the true power of meta-analytic models for dependent effect sizes. Lastly, the study compares the Type I error rate and power of several common models, finding that tests using robust variance estimation offer better Type I error calibration than tests with model-based variance estimation.
Meta-analytic models for dependent effect sizes have grown increasingly sophisticated over the last few decades, which has created challenges for a priori power calculations. We introduce power approximations for tests of average effect sizes based upon several common approaches for handling dependent effect sizes. In a Monte Carlo simulation, we show that the new power formulas can accurately approximate the true power of meta-analytic models for dependent effect sizes. Lastly, we investigate the Type I error rate and power for several common models, finding that tests using robust variance estimation provide better Type I error calibration than tests with model-based variance estimation. We consider implications for practice with respect to selecting a working model and an inferential approach.

Authors

I am an author on this paper
Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

4.2
Not enough ratings

Secondary Ratings

Novelty
-
Significance
-
Scientific rigor
-
Rate this paper

Recommended

Review Substance Abuse

Linking Big Five Personality Domains and Facets to Alcohol (Mis)Use: A Systematic Review and Meta-Analysis

P. Priscilla Lui, Michael Chmielewski, Mayson Trujillo, Joseph Morris, Terri D. Pigott

Summary: The study found that conscientiousness and agreeableness were negatively correlated with alcohol consumption, risky drinking, and negative consequences. Narrowband traits such as deliberation, dutifulness, compliance, and straightforwardness were uniquely associated with alcohol use outcomes. Extraversion, specifically excitement seeking, was correlated with alcohol consumption, while neuroticism, specifically impulsiveness and angry hostility, was correlated with negative drinking-related consequences.

ALCOHOL AND ALCOHOLISM (2022)

Review Psychology, Mathematical

Examining the relations between spatial skills and mathematical performance: A meta-analysis

Kinnari Atit, Jason Richard Power, Terri Pigott, Jihyun Lee, Elyssa A. Geer, David H. Uttal, Colleen M. Ganley, Sheryl A. Sorby

Summary: Recent research has focused on the relationship between spatial skills and mathematical skills. The findings show a positive moderate association between the two skills, with gender and grade-level not significantly affecting this relationship. Fluid reasoning and verbal skills mediate the relationship between spatial skills and mathematical skills, suggesting a unique connection between the two.

PSYCHONOMIC BULLETIN & REVIEW (2022)

Article Education & Educational Research

A Meta-Analysis of Mathematics Word-Problem Solving Interventions for Elementary Students Who Evidence Mathematics Difficulties

Jonte A. Myers, Bradley S. Witzel, Sarah R. Powell, Hongli Li, Terri D. Pigott, Yan P. Xin, Elizabeth M. Hughes

Summary: This study conducted a meta-analysis of 52 studies to examine the effect of interventions on improving word-problem performance in elementary school students with mathematics difficulties. The results showed a significant and positive effect size, with several factors explaining the heterogeneity in effects across studies.

REVIEW OF EDUCATIONAL RESEARCH (2022)

Review Education & Educational Research

Promoting Knowledge Accumulation About Intervention Effects: Exploring Strategies for Standardizing Statistical Approaches and Effect Size Reporting

Joseph A. Taylor, Terri Pigott, Ryan Williams

Summary: This article discusses statistical approaches to improve the precision and comparability of effect sizes in education research through meta-analyses. It focuses on a standardized mean difference effect size adjusted for baseline differences and total variance. The article highlights the utility of covariate adjustment and the importance of standardizing effects on total variance accounting for variation at multiple levels.

EDUCATIONAL RESEARCHER (2022)

Review Criminology & Penology

Childhood Polyvictimization and Associated Health Outcomes: A Systematic Scoping Review

NaeHyung Lee, Terri Deocampo Pigott, Ashley Watson, Katherine Reuben, Kathryn O'Hara, Greta Massetti, Xiangming Fang, Shannon Self-Brown

Summary: This scoping review examines the literature on polyvictimization and health outcomes, highlighting the varied constructions of polyvictimization and identifying gaps in knowledge. The findings emphasize the need for a standardized definition of polyvictimization and suggest specific health outcomes that should be investigated further. The study also underscores the importance of resilience and coping education for childhood polyvictims.

TRAUMA VIOLENCE & ABUSE (2023)

Review Psychology, Developmental

Considering Instructional Contexts in AAC Interventions for People with ASD and/or IDD Experiencing Complex Communicative Needs: a Single-Case Design Meta-analysis

Jay B. Ganz, James E. Pustejovsky, Joe Reichle, Kimberly J. Vannest, Margaret Foster, April N. Haas, Lauren M. Pierson, Sanikan Wattanawongwan, Armando Bernal, Man Chen, Rachel Skov, S. D. Smith

Summary: Communication is an essential skill for children with autism or intellectual and developmental disabilities. This meta-analysis investigated the effectiveness of augmentative and alternative communication interventions for these children. The results showed that these interventions were effective in improving communication skills, but instructional context variables did not significantly predict intervention effectiveness.

REVIEW JOURNAL OF AUTISM AND DEVELOPMENTAL DISORDERS (2023)

Review Audiology & Speech-Language Pathology

Participant characteristics predicting communication outcomes in AAC implementation for individuals with ASD and ;DD: a systematic review and meta-analysis

J. B. Ganz, James E. Pustejovsky, Joe Reichle, Kimberly J. Vannest, Margaret Foster, Lauren M. Pierson, Sanikan Wattanawongwan, Armando J. Bernal, Man Chen, April N. Haas, Ching-Yi Liao, Mary Rose Sallese, Rachel Skov, S. D. Smith

Summary: This meta-analysis explores the relationship between communication outcomes and participant characteristics in single-case design studies of augmentative and alternative communication (AAC) interventions. The results indicate positive effects of AAC interventions on communication outcomes, although there is considerable heterogeneity. Moderator analyses suggest that the effectiveness of interventions does not significantly vary across diagnoses, age, communication modes, productive repertoires, and imitation skills.

AUGMENTATIVE AND ALTERNATIVE COMMUNICATION (2023)

Article Psychology, Educational

Meta-Analysis of the Relationship Between Self-Critical Perfectionism and Depressive Symptoms: Comparison Between Asian American and Asian International College Students

Han Na Suh, Terri Pigott, Kenneth G. Rice, Don E. Davis, Andrea C. Andrade

Summary: A body of research has explored the relationship between perfectionism and mental health indicators. Self-critical perfectionism, a component of perfectionism, is seen as negative in the context of mental health. It is important to examine the generalizability of these findings across cultures and subcultures, as research informs educational and clinical practices.

JOURNAL OF COUNSELING PSYCHOLOGY (2023)

Review Audiology & Speech-Language Pathology

Systematic Review of Variables Related to Instruction in Augmentative and Alternative Communication Implementation: Group and Single-Case Design

Joe Reichle, James E. Pustejovsky, Kimberly J. Vannest, Margaret Foster, Lauren M. Pierson, Sanikan Wattanawongwan, Man Chen, Marcus C. Fuller, April N. Haas, Bethany H. Bhat, Mary Rose Sallese, S. D. Smith, Valeria Yllades, Daira Rodriguez, Amara Yoro, J. B. Ganz

Summary: This article provides a systematic review and analysis of group and single-case studies on augmentative and alternative communication (AAC) intervention with school-aged individuals with autism spectrum disorder (ASD) and/or intellectual/developmental disabilities resulting in complex communication needs (CCNs). The findings reveal that participant characteristics and pivotal skills are not adequately reported in both types of studies. Furthermore, group studies tend to utilize clinical settings while single-case studies prefer instructional methods associated with behavioral approaches.

AMERICAN JOURNAL OF SPEECH-LANGUAGE PATHOLOGY (2023)

Article Psychology, Multidisciplinary

Comparing Random Effects Models, Ordinary Least Squares, or Fixed Effects With Cluster Robust Standard Errors for Cross-Classified Data

Young Ri Lee, James E. Pustejovsky

Summary: Cross-classified random effects modeling (CCREM) is a common approach for analyzing cross-classified data. However, ordinary least squares regression with cluster robust variance estimators (OLS-CRVE) or fixed effects regression with CRVE (FE-CRVE) may be more appropriate approaches when the focus is on Level 1 regression coefficients. We compared the performance of CCREM, OLS-CRVE, and FE-CRVE in different model conditions. CCREM out-performed the alternative approaches when its assumptions are met, but OLS-CRVE and FE-CRVE provided similar or better performance when assumptions were violated.

PSYCHOLOGICAL METHODS (2023)

Review Education & Educational Research

The Effects of Co-Teaching and Related Collaborative Models of Instruction on Student Achievement: A Systematic Review and Meta-Analysis

Mikkel Helding Vembye, Felix Weiss, Bethany Hamilton Bhat

Summary: This systematic review and meta-analysis examined the effects of co-teaching and collaborative models of instruction on students' academic achievement. The study found that collaborative instruction had a moderate, positive effect on students' academic achievement compared to single-taught controls, regardless of the teachers' qualifications.

REVIEW OF EDUCATIONAL RESEARCH (2023)

Article Substance Abuse

Differences in medicaid expansion effects on buprenorphine treatment utilization by county rurality and income: A pharmacy data claims analysis from 2009-2018

Olivia K. Golan, Flora Sheng, Andrew W. Dick, Mark Sorbero, Daniel J. Whitaker, Barbara Andraka-Christou, Therese Pigott, Adam J. Gordon, Bradley D. Stein

Summary: This study examines the impact of the Affordable Care Act Medicaid expansion on the initiation rates of buprenorphine, a medication used to treat opioid use disorder. The results show that the expansion reduced income-related disparities in urban counties but had no significant effect in rural counties.

DRUG AND ALCOHOL DEPENDENCE REPORTS (2023)

No Data Available