Using metaheuristics to optimize the combination of classifier and cluster ensembles

We investigate how to make a simpler version of an existing algorithm, named (CE)-E-3, from Consensus between Classification and Clustering Ensembles, more user-friendly by automatically tuning its main parameters with the use of metaheuristics. In particular, (CE)-E-3 based on a Squared Loss function, (CE)-E-3-SL, assumes an optimization procedure that takes as input class membership estimates from existing classifiers, as well as a similarity matrix from a cluster ensemble operating solely on the new target data, to provide a consolidated classification of the target data. To do so, two parameters have to be defined a priori, namely: the relative importance of classifier and cluster ensembles and the number of iterations of the algorithm. In some practical applications, these parameters can be optimized via time consuming grid search approaches based on cross-validation procedures. This paper shows that seven metaheuristics for parameter optimization yield classifiers as accurate as those obtained from grid search, but taking half the running time. More precisely, and by assuming a trade-off between user-friendliness and accuracy, experiments performed on twenty real-world datasets suggest that CMA-ES, DE, and SaDE are the best alternatives to optimize the (CE)-E-3-SL parameters.