Article
Engineering, Electrical & Electronic
Ilyas El Jaafari, Ayoub Ellahyani, Said Charfi
Summary: The proposed parametric rectified nonlinear function unit (PRenu) is similar to Relu but differentiates by providing a non-linear transformation for positive values and improves CNN convergence speed and accuracy.
SIGNAL IMAGE AND VIDEO PROCESSING
(2021)
Article
Computer Science, Artificial Intelligence
Donghun Yang, Kien Mai Ngoc, Iksoo Shin, Myunggwon Hwang
Summary: Activation functions play a crucial role in deep learning, and the rectified linear unit (ReLU) has become the most widely used activation function due to its ability to address the vanishing gradient issue. However, ReLU suffers from the dying ReLU problem and bias shift effect, limiting its ability to utilize negative values effectively. To tackle this problem, numerous variants of ReLU have been proposed. In this study, Dynamic Parametric ReLU (DPReLU) is introduced, which allows for dynamic control of the overall shape of ReLU through four learnable parameters. The parameters of DPReLU are determined through training, making it more suitable and flexible for each model and dataset. Additionally, an appropriate and robust weight initialization method for DPReLU is proposed. Experimental results on various image datasets demonstrate that DPReLU and the weight initialization method lead to faster convergence and better accuracy compared to the original ReLU and previous variants.
INTERNATIONAL JOURNAL OF COMPUTATIONAL INTELLIGENCE SYSTEMS
(2023)
Article
Computer Science, Software Engineering
Bekhzod Olimov, Sanjar Karshiev, Eungyeong Jang, Sadia Din, Anand Paul, Jeonghong Kim
Summary: This paper explores different weight initialization strategies and proposes a weight initialization-based ReLU activation function, which performs better on Fashion-MNIST and CIFAR-10 databases.
CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE
(2021)
Article
Computer Science, Hardware & Architecture
Bo Li, Guoyong Shi
Summary: This brief presents the design of a CMOS ultra low-power rectified linear unit (ReLU) operating in weak inversion and validates its working mechanism through simulation and analysis. The proposed ReLU circuit is then applied to an analog CMOS-memristive artificial neural network (ANN) for pattern recognition, achieving comparable functionality to a software implementation.
INTEGRATION-THE VLSI JOURNAL
(2022)
Article
Geochemistry & Geophysics
Shuwen Xu, Hongtao Ru, Dongchen Li, Penglang Shui, Jian Xue
Summary: This article presents a method for effectively classifying different marine floating small targets in strong clutter background. It is based on block-whitened time-frequency spectrogram and pre-trained convolution neural network (CNN). The proposed method suppresses clutter and converts it to approximately noisy background, reducing its impact on classification. It extracts time-frequency spectrogram of targets from block-whitened echoes, provides more information. The pre-trained CNN using block-whitened time-frequency spectrograms achieves higher performance on the measured dataset compared to competitors.
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING
(2023)
Article
Computer Science, Information Systems
Akriti Bhusal, Abeer Alsadoon, P. W. C. Prasad, Nada Alsalami, Tarik A. Rashid
Summary: Each stage of sleep has an impact on human health, and insufficient sleep can lead to sleep disorders. The research aims to improve the accuracy and learning time of the Convolutional Neural Network Classifier by using a modified Orthogonal Convolutional Neural Network and a modified Adam optimisation technique. The proposed system, called Enhanced Sleep Stage Classification system (ESSC), achieves better classification accuracy and convergence speed compared to existing solutions.
MULTIMEDIA TOOLS AND APPLICATIONS
(2022)
Article
Engineering, Multidisciplinary
Ganapathi Ammasai Sengodan
Summary: This work presents a novel method to predict the mechanical responses of arbitrary microstructures using deep learning, generating two-phase microstructural images, quantifying them using the two-point statistical homogenisation scheme, and projecting microstructures and stress-strain data into lower order orthogonal spaces by principal component analysis to minimize computational efforts. By using convolutional neural networks to learn reduced order statistically homogeneous microstructures and stress-strain data, the study successfully predicts the mechanical responses of randomly generated two-phase fibre reinforced plastic (FRP) composite microstructures with better accuracy and minimal computational effort.
COMPOSITES PART B-ENGINEERING
(2021)
Article
Chemistry, Multidisciplinary
Xiaofeng Wang, Xiuyan Liu, Jinlong Wang, Xiaoyun Xiong, Suhuan Bi, Zhaopeng Deng
Summary: Rolling bearings are crucial for safe and efficient operation of machinery, but faults can lead to downtime and economic costs. A model based on improved grey wolf optimization and improved 1DCNN is proposed to extract fault features. Experimental results show the effectiveness and high recognition accuracy of the model.
APPLIED SCIENCES-BASEL
(2022)
Article
Computer Science, Information Systems
Amani Ali Ahmed Ali, Suresha Mallaiah
Summary: Text recognition in Arabic handwritten scripts is a challenging research field. In this study, authors propose a deep learning model for efficiently recognizing Arabic handwritten scripts. The model combines Support Vector Machine (SVM) and Convolutional Neural Network (CNN) classifiers for classification and feature extraction, and addresses the issue of over-fitting using dropout technique. The proposed model achieves favorable results in text recognition accuracy, as demonstrated through testing on multiple databases.
JOURNAL OF KING SAUD UNIVERSITY-COMPUTER AND INFORMATION SCIENCES
(2022)
Article
Computer Science, Theory & Methods
Ibrahim A. Atoum
Summary: A convolutional neural network (CNN) is a type of artificial neural network used for various applications. Activation functions, such as Rectified Linear Units (ReLU), are used to determine neuron activation. ReLU overcomes the vanishing gradient problem of older activation functions and has lower computational cost. However, it suffers from the dying problem, which can be mitigated by readjusting the loss function during training.
INTERNATIONAL JOURNAL OF ADVANCED COMPUTER SCIENCE AND APPLICATIONS
(2023)
Article
Geochemistry & Geophysics
Ju Zhang, Qingwu Hu, Jiayuan Li, Mingyao Ai
Summary: This article proposes a new method using GPS trajectories to generate sample sets for the extraction of multi-level urban roads from high-resolution remote sensing imagery. By eliminating the manual labeling work, the method achieves a higher harmonic mean of precision and recall compared to traditional methods of road extraction from single data sources.
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING
(2021)
Article
Engineering, Electrical & Electronic
Wei Hou, Xian Tao, De Xu
Summary: This article proposes a novel weak scratch detection method for optical components, which achieves high accuracy detection of weak scratches on the surface by incorporating prior knowledge into the model, outperforming other methods.
IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT
(2021)
Article
Computer Science, Cybernetics
Selvam Lakshmanan, Uma Maheswari Gnaniyan Ponnusamy, Senthilkumar Andi
Summary: Denial of Service (DoS) attacks are a major concern for cloud service providers due to their complexity. This article introduces a novel Chaos-based HGSO-WIB-ReLU framework to detect different types of DoS attacks. The framework incorporates a WIB-ReLU activation function in a convolutional neural network architecture to provide effective training and minimize false positives. Experimental analysis on benchmark datasets demonstrates the effectiveness of the proposed framework with high accuracy rates.
CYBERNETICS AND SYSTEMS
(2023)
Article
Medicine, General & Internal
Akira Takekawa, Masayuki Kajiura, Hiroya Fukuda
Summary: This paper focuses on the theoretical analysis of deep learning using the rectified linear unit (ReLU) activation function. Increasing the number of layers improves the approximation accuracy of the curved surface, while increasing the number of neurons cannot improve the results obtained. These results illustrate the functions of layers and neurons in deep learning with ReLU.
CUREUS JOURNAL OF MEDICAL SCIENCE
(2021)
Article
Automation & Control Systems
Muhamad Dwisnanto Putro, Laksono Kurnianggoro, Kang-Hyun Jo
Summary: Face detection is crucial for the development of face recognition, expression, tracking, and classification. While conventional methods have accuracy constraints on challenging conditions, CNN methods show high performances, though requiring expensive hardware. The proposed CNN-based face detector achieves state-of-the-art performance on benchmark datasets and runs efficiently on a CPU.
IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS
(2021)
