Accounting for Heteroskedasticity Resulting from Between-Group Differences in Multilevel Models

Homogeneity of variance (HOV) is a well-known but often untested assumption in the context of multilevel models (MLMs). However, depending on how large the violation is, how different group sizes are, and the variance pairing, standard errors can be over or underestimated even when using MLMs, resulting in questionable inferential tests. We evaluate several tests (e.g., the H statistic, Breusch Pagan, Levene's test) that can be used with MLMs to assess violations of HOV. Although the traditional robust standard errors used with MLMs require at least 50 clusters to be effective, we assess a robust standard error adjustment (i.e., the CR2 estimator) that can be used even with a few clusters. Findings are assessed using a Monte Carlo simulation and are further illustrated using an applied example. We show that explicitly modeling the heterogenous variance structures or using the CR2 estimator are both effective at ameliorating the issues associated with the fixed effects of the regression model related to violations of HOV resulting from between-group differences.

Automated summary

Homogeneity of variance (HOV) is an important but often untested assumption in multilevel models (MLMs). This study evaluates several tests to assess violations of HOV, such as H statistic, Breusch Pagan, Levene's test. The findings suggest that explicitly modeling heterogenous variance structures or using the CR2 estimator can effectively address the issues caused by between-group differences related to violations of HOV.