Uncertainty Quantification with Experts: Present Status and Research Needs

Expert elicitation is deployed when data are absent or uninformative and critical decisions must be made. In designing an expert elicitation, most practitioners seek to achieve best practice while balancing practical constraints. The choices made influence the required time and effort investment, the quality of the elicited data, experts' engagement, the defensibility of results, and the acceptability of resulting decisions. This piece outlines some of the common choices practitioners encounter when designing and conducting an elicitation. We discuss the evidence supporting these decisions and identify research gaps. This will hopefully allow practitioners to better navigate the literature, and will inspire the expert judgment research community to conduct well powered, replicable experiments that properly address the research gaps identified.

Automated summary

Expert elicitation is used when data is lacking and important decisions need to be made. When designing expert elicitation, practitioners aim to balance best practices with practical constraints. The choices made impact time and effort investment, data quality, expert engagement, result defensibility, and decision acceptability.