Journal
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
Volume 31, Issue 10, Pages 4217-4228Publisher
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TNNLS.2019.2952864
Keywords
Neurons; Adaptation models; Adaptive learning; Training; Robustness; Convergence; Modeling; Adaptive-sparse restricted Boltzmann machine (RBM); convergence analysis; deep belief network (DBN); partial least square (PLS)-based regression fine-tuning; robust structure
Categories
Funding
- Key Project of National Natural Science Foundation of China [61533002]
- National Natural Science Foundation of China [61703011, 61673229]
- Major Project for New Generation Artificial Intelligence [2018AAA0101600]
- National Science and Technology Major Project [2018ZX07111005]
Ask authors/readers for more resources
Deep belief network (DBN) is an efficient learning model for unknown data representation, especially nonlinear systems. However, it is extremely hard to design a satisfactory DBN with a robust structure because of traditional dense representation. In addition, backpropagation algorithm-based fine-tuning tends to yield poor performance since its ease of being trapped into local optima. In this article, we propose a novel DBN model based on adaptive sparse restricted Boltzmann machines (AS-RBM) and partial least square (PLS) regression fine-tuning, abbreviated as ARP-DBN, to obtain a more robust and accurate model than the existing ones. First, the adaptive learning step size is designed to accelerate an RBM training process, and two regularization terms are introduced into such a process to realize sparse representation. Second, initial weight derived from AS-RBM is further optimized via layer-by-layer PLS modeling starting from the output layer to input one. Third, we present the convergence and stability analysis of the proposed method. Finally, our approach is tested on Mackey-Glass time-series prediction, 2-D function approximation, and unknown system identification. Simulation results demonstrate that it has higher learning accuracy and faster learning speed. It can be used to build a more robust model than the existing ones.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available