SinLU: Sinu-Sigmoidal Linear Unit

Average biased ReLU based CNN descriptor for improved face retrieval

(2021) Shiv Ram Dubey et al.   MULTIMEDIA TOOLS AND APPLICATIONS

TanhExp: A smooth activation function with high convergence speed for lightweight neural networks

(2021) Xinyu Liu et al.   IET Computer Vision

A survey on modern trainable activation functions

(2021) Andrea Apicella et al.   NEURAL NETWORKS

Logish: A new nonlinear nonmonotonic activation function for convolutional neural network

(2021) Hegui Zhu et al.   NEUROCOMPUTING

Learnable Leaky ReLU (LeLeLU): An Alternative Accuracy-Optimized Activation Function

(2021) Andreas Maniatopoulos et al.   Information (Switzerland)

Activated Gradients for Deep Neural Networks

(2021) Mei Liu et al.   IEEE Transactions on Neural Networks and Learning Systems

“SPOCU”: scaled polynomial constant unit activation function

(2020) Jozef Kiseľák et al.   NEURAL COMPUTING & APPLICATIONS

ReLTanh: An activation function with vanishing gradient resistance for SAE-based DNNs and its application to rotating machinery fault diagnosis

(2019) Xin Wang et al.   NEUROCOMPUTING

Sigmoid-weighted linear units for neural network function approximation in reinforcement learning

(2018) Stefan Elfwing et al.   NEURAL NETWORKS

Deep Residual Bidir-LSTM for Human Activity Recognition Using Wearable Sensors

(2018) Yu Zhao et al.   MATHEMATICAL PROBLEMS IN ENGINEERING

LSTM: A Search Space Odyssey

(2017) Klaus Greff et al.   IEEE Transactions on Neural Networks and Learning Systems

A linear approach for sparse coding by a two-layer neural network

(2015) Alessandro Montalto et al.   NEUROCOMPUTING