Hierarchical ensemble of Extreme Learning Machine

Extreme Learning Machine (ELM), which is proposed for generalized single-hidden layer feedforward neural networks, has become a popular research topic due to its unique characteristics. However, the random nature inherent in ELM's hidden layer results in unstable performance and a large number of hidden neurons is required, making the risk of overfitting increased. In this paper, we propose a simple but effective ensemble approach, called Hierarchical Ensemble of Extreme Learning Machine (HE-ELM), to improve ELM. To encourage the diversity of component ELMs, two strategies are taken into account, namely, the sparse connection to component ELMs and feature bagging. The resulting architecture is able to integrate both representation learning and ensemble learning with relatively fewer parameters and consists of independent component ELMs, making it easy to implement, train, and apply in practice. We compare results of the proposed HE-ELM with existing methods for 22 classification problems, showing that HEELM is able to achieve significant improvement in terms of classification accuracy, with a reduced risk of overfitting the training data. (C) 2018 Elsevier B.V. All rights reserved.