Sum of ranking differences compares methods or models fairly

This review covers a novel approach to comparing methods, based on the sum of ranking differences (SRD). Many method-comparison studies suffer from ambiguity or from comparisons not being quite fair. This problem can be avoided if there are differences between ideal and actual rankings. The absolute values of differences for the ideal and actual ranking are summed up and the procedure is repeated for each (actual) method. The SRD values obtained such a way order the methods simply. If the ideal ranking is not known, it can be replaced by the average (maximum or minimum of all methods or by a known sequence). SRD corresponds to the principle of parsimony and provides an easy tool to evaluate the methods: the smaller the sum the better the method. Models and other items can be similarly ranked. Validation can be carried out using simulated random numbers for comparison: an empirical histogram (bootstrap-like) shows whether the SRD values are far from random. Two case studies (clustering of HPLC columns and prediction of retention data) illustrate and validate the applicability of this novel approach to comparing methods. The technique is entirely general; it can be used in different fields (e.g., for stationary-phase (column) selection in chromatography, model and descriptor selection, comparing analytical and chemometric techniques, determination of panel consistency, etc.). The only prerequisite is that the data can be arranged in matrix form without empty cells. (C) 2009 Elsevier Ltd. All rights reserved.