Characterizing multivariate calibration tradeoffs (bias, variance, selectivity, and sensitivity) to select model tuning parameters

Common methods used to form multivariate calibration models are partial least squares, principal component regression, and ridge regression. There are many aspects to forming a calibration model. The focus in this paper is determining good values of respective meta-parameters (tuning parameters). In most situations, some form of cross-validation and the resultant root mean square error of cross-validation (RMSECV) values are evaluated. The RMSECV values can be thought of as a prediction accuracy (bias) measure and does not isolate variance information. The methods of partial least squares, principal component regression, and ridge regression (and others) are biased regression methods, and hence, there is a bias/variance tradeoff that should be considered in determining tuning parameter values. Presented in this paper are new combinations of model accuracy and fit that can be plotted with a model variance indicator for selecting appropriate tuning parameter values. For the spectroscopic and industrial data sets evaluated, these new combinations appear to remove ambiguities that can occur in RMSECV plots. Essentially, the user's preference for the degree of balance between bias and variance ultimately decides the tuning parameter selection and the underlying tradeoff between model selectivity and sensitivity. From the analysis of the selectivity/sensitivity tradeoff, a new definition of model selectivity is proposed that is able to discern when or when not a model vector deviates from the target orthogonal net analyte signal model vector as the well as the degree of deviation. A new merit is also proposed that simultaneously evaluates model selectivity and sensitivity. Copyright (c) 2013 John Wiley & Sons, Ltd. A new model merit is presented for automatic selection of appropriate tuning parameter values with a good bias/variance balance. The bias/variance balance is related to the underlying model selectivity/sensitivity tradeoff. A new definition of model selectivity is proposed that is able to discern when a model vector deviates from the ideal orthogonal net analyte signal model vector as the well as the degree of deviation. A new merit is also proposed that simultaneously evaluates model selectivity and sensitivity.