Item Parameter Recovery: Sensitivity to Prior Distribution

Marginal maximum likelihood, a common estimation method for item response theory models, is not inherently a Bayesian procedure. However, due to estimation difficulties, Bayesian priors are often applied to the likelihood when estimating 3PL models, especially with small samples. Little focus has been placed on choosing the priors for marginal maximum estimation. In this study, using sample sizes of 1,000 or smaller, not using priors often led to extreme, implausible parameter estimates. Applying prior distributions to the c-parameters alleviated the estimation problems with samples of 500 or more; for the samples of 100, priors on both the a-parameters and c-parameters were needed. Estimates were biased when the mode of the prior did not match the true parameter value, but the degree of the bias did not depend on the strength of the prior unless it was extremely informative. The root mean squared error (RMSE) of the a-parameters and b-parameters did not depend greatly on either the mode or the strength of the prior unless it was extremely informative. The RMSE of the c-parameters, like the bias, depended on the mode of the prior for c.

Automated summary

Marginal maximum likelihood is a popular estimation method in item response theory models, and Bayesian priors are often applied to likelihood in 3PL models when dealing with small sample sizes. Choosing appropriate priors for marginal maximum estimation has been overlooked. In this study, not using priors resulted in extreme and implausible parameter estimates for sample sizes of 1,000 or smaller. Applying priors to the c-parameters alleviated estimation problems for sample sizes of 500 or more, while both a-parameters and c-parameters needed priors for samples of 100. Bias was observed when the mode of the prior did not match the true parameter value, but the strength of the prior did not significantly affect the bias unless it was extremely informative. The root mean squared error (RMSE) of the a-parameters and b-parameters did not heavily depend on the mode or strength of the prior unless it was extremely informative. The RMSE of the c-parameters, similar to the bias, depended on the mode of the prior for c.