Not adding up: free ridership and spillover calculations in energy efficiency evaluations

A key element of evaluation, measurement, and verification (EM&V) studies for energy efficiency programs involves estimation of net energy savings that account for free ridership, spillover, and induced market effects. The existing literature recognizes these effects to be significant and provides detailed guidelines to estimate them. However, there appears to be a disconnect between these guidelines and field evaluations conducted in practice. Our meta-analysis of 120 studies from 2006 to 2018 indicates that most free ridership and spillover estimates are based on survey results and expressed in percentage terms. We note that simply adding these percentages numerically without converting them into a common unit is inaccurate and obscures a program's true impact. Additionally, there exists wide variations in nomenclature, classification, and methodologies adopted to estimate these metrics across programs and jurisdictions. Our scatterplot analysis of the reviewed EM&V reports indicates that with few exceptions, free ridership and spillover do not necessarily offset each other. We propose an alternative approach to estimate free ridership and spillover in energy units with costs in dollar terms, e.g., as the difference between a program participant's total willingness-to-pay and the total financial impact of the program's existence. We also feel that a consistent, transparent, and reliable evaluation methodology to estimate free ridership and spillover effects across programs and jurisdictions based on randomized or quasi-experimental designs will not only improve accuracy but will also have better comparability for informed policy decisions in future.