Distance-Based Subset Selection for Benchmarking in Evolutionary Multi/Many-Objective Optimization

A number of real-world problems involve extremization of multiple conflicting objectives, referred to as multiobjective optimization problems. Multiobjective evolutionary algorithms (MOEAs) have been widely adopted to obtain Pareto front (PF) approximation for such problems. An indispensable step in development and evaluation of MOEAs is benchmarking, which involves comparisons with peer algorithms using performance metrics, such as hypervolume (HV) and inverted generational distance (IGD). However, the de-facto practice is to use the final population of an algorithm for evaluating these metrics even though a better PF approximation may exist within the archive of all evaluated solutions. In a recent study, a distance-based subset selection (DSS) method was discussed for selecting prespecified number of solutions from an archive for benchmarking. This letter aims to contribute toward this direction in two ways. First is to develop a theoretical understanding of DSS and reveal some of its interesting and desirable properties. These include conditional equivalence to optimal HV/IGD subset selection, inclusion of PF extremities and invariance to convexity/concavity and orientation of the PF. Secondly, we present numerical experiments on problems with regular and irregular PFs up to ten objectives, and compare the approach with other selection techniques to demonstrate its potential benefits. The results clearly indicate the importance of considering the archive of solutions and appropriate selection mechanisms, in particular for problems with irregular PFs, to avoid misjudgments about relative performances. With increasing emphasis on tackling such problems in the field, we believe that this observation and analysis is timely and significant, not only for benchmarking, but also for subsequent improvements in decision-making and algorithm design.