Attribute reduction via local conditional entropy

In rough set theory, the concept of conditional entropy has been widely accepted for studying the problem of attribute reduction. If a searching strategy is given to find reduct, then the value of conditional entropy can also be used to evaluate the significance of the candidate attribute in the process of searching. However, traditional conditional entropy is used to characterize the relationship between conditional attributes and decision attribute in terms of all samples in data, it does not take such relationship with specific samples (samples with same label) into account. To fill such a gap, a new form of conditional entropy which is termed as Local Conditional Entropy is proposed. Furthermore, based on some important properties about local conditional entropy studied, local conditional entropy based attribute reduction is defined. Immediately, an ensemble strategy is introduced into the heuristic process for searching reduct, which is realized by the significance based on local conditional entropy. Finally, the experimental results over 18 UCI data sets show us that local conditional entropy based attribute reduction is superior to traditional conditional entropy based attribute reduction, the former may provide us attributes with higher classification accuracies. In addition, if local conditional entropy is regarded as the measurement in online feature selection, then it not only offers us better classification performance, but also requires lesser elapsed time to complete the process of online feature selection. This study suggests new trends for considering attribute reduction and provides guidelines for designing new measurements and related algorithms.