Measuring the Diversity of a Test Set With Distance Entropy

Most existing metrics that we call white-box metrics, such as coverage metrics, require white-box information, like program structure information, and historical runtime information, to evaluate the fault detection capability of a test set. In practice, such white-box information is usually unavailable or difficult to obtain, which means they often cannot be used. In this paper, we propose a black-box metric, distance entropy, based on the diversification idea behind many published diversity-based techniques. Distance entropy provides a possible solution for test set evaluation when white-box information is not available. The empirical study illustrates that distance entropy can effectively evaluate test sets if the distance metric between tests is well defined. Meanwhile, distance entropy outperforms simple diversity metrics without increasing time complexity.