Article
Computer Science, Artificial Intelligence
Jin-Hui Wu, Shao-Qun Zhang, Yuan Jiang, Zhi-Hua Zhou
Summary: Neural network models consist of network architecture and neuron model. While there are many studies on network architectures, limited progress has been made on neuron models, except for the MP neuron model developed in 1943 and the spiking neuron model developed in the 1950s. In this paper, a new bio-plausible neuron model called Flexible Transmitter (FT) model is proposed, which shows promising behaviors on temporal-spatial signals when embedded in a common feedforward network architecture. The paper aims to theoretically understand the properties of the FT network (FTNet) and demonstrates that it is a universal approximator with efficient approximation complexity compared to commonly-used real-valued neural networks.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2023)
Article
Computer Science, Artificial Intelligence
Ingo Guehring, Mones Raslan
Summary: This study investigates the necessary and sufficient complexity of neural networks to approximate functions from different smoothness spaces under the restriction of encodable network weights. By providing a unifying framework for constructing approximate partitions of unity with fairly general activation functions, almost optimal upper bounds in higher-order Sobolev norms were derived. This work advances the theory of approximating solutions of partial differential equations by neural networks.
Article
Computer Science, Information Systems
Bartosz Puchalski
Summary: The paper presents research on the approximation of variable-order fractional operators using recurrent neural networks with GRU cells, which can satisfactorily approximate targeted variable-order fractional operators with minor modeling errors.
Article
Mathematics, Applied
Jonathan W. Siegel, Jinchao Xu
Summary: This study investigates the approximation properties of shallow neural networks and the relationship between the activation function and the dimension and smoothness of the underlying function to be approximated. The study reveals that shallow neural networks with the ReLUk activation function achieve an improved approximation rate in L2 norm, independent of the dimension. Additionally, shallow neural networks with cosine activation function also obtain an improved approximation rate in the spectral Barron space.
APPLIED AND COMPUTATIONAL HARMONIC ANALYSIS
(2022)
Article
Computer Science, Theory & Methods
Xia Liu
Summary: Constructing deep neural networks with three hidden layers using a sigmoidal activation function is the main focus of this paper, aiming to approximate smooth and sparse functions. The paper proves that by controlling the number of free parameters, these constructed deep networks can achieve the optimal approximation rate for both smooth and sparse functions. Furthermore, it is shown that neural networks with three hidden layers can overcome the saturation phenomenon observed in some architectures.
JOURNAL OF COMPLEXITY
(2023)
Article
Engineering, Multidisciplinary
Lin Xu, Xiangyong Cao, Jing Yao, Zheng Yan
Summary: In this paper, we propose an Orthogonal Super Greedy learning (OSGL) method for hidden neurons selection in feedforward neural networks. The method addresses the issue of generation performance and computational complexity being affected by irrelevant hidden variables. Theoretical analyses and empirical results demonstrate that the proposed method can achieve optimal learning rate and produce excellent generalization performance with a sparse and compact feature representation.
IEEE TRANSACTIONS ON NETWORK SCIENCE AND ENGINEERING
(2022)
Article
Automation & Control Systems
Haitao Liu, Yew-Soon Ong, Xiaomo Jiang, Xiaofang Wang
Summary: Deep kernel learning (DKL) integrates neural networks and Gaussian processes to build an end-to-end hybrid model, while deep latent-variable kernel learning (DLVKL) model achieves regularized representation through stochastic encoding and improves performance through neural stochastic differential equation (NSDE) and hybrid prior.
IEEE TRANSACTIONS ON CYBERNETICS
(2022)
Article
Computer Science, Artificial Intelligence
Jin Li, Zilong Liu
Summary: This paper proposes a low-dimensional visual representation convolution neural network (LVR-CNN) for efficient post-transform-based image compression in high-resolution imaging of an on-orbit optical camera. The LVR-CNN transforms the wavelet domain from a large-scale representation to a new wavelet version with a small scale, optimizing compression performance and calculation efficiency. Experimental results show that the proposed LVR-CNN post-transform-based compression method outperforms conventional methods by increasing the peak-signal-noise-ratio (PSNR) by 1.2 to 2.7 dB, indicating its efficiency for remote sensing images.
APPLIED SOFT COMPUTING
(2021)
Article
Engineering, Electrical & Electronic
Gen Li, Jie Ding
Summary: Multi-layer feedforward networks can approximate nonlinear functions well, and overparameterized deep neural networks may not suffer from overfitting when the number of neurons and learning parameters rapidly grow. This paper shows that a class of variation-constrained regression neural networks can achieve a near-parametric rate for an arbitrarily small constant. The result provides insight into the benign overparameterization phenomenon and suggests that the number of trainable parameters may not be a suitable complexity measure for classical regression models.
IEEE TRANSACTIONS ON SIGNAL PROCESSING
(2023)
Article
Physics, Fluids & Plasmas
Elham Kiyani, Steven Silber, Mahdi Kooshkbaghi, Mikko Karttunen
Summary: This paper presents data-driven architectures based on machine learning algorithms for discovering nonlinear equations of motion for phase-field models. The experimental results show that we can effectively learn the time derivatives of the field and use the data-driven partial differential equations (PDEs) to propagate the field in time, achieving results in good agreement with the original PDEs.
Article
Computer Science, Artificial Intelligence
Denis Belomestny, Alexey Naumov, Nikita Puchkin, Sergey Samsonov
Summary: This paper investigates the approximation properties of deep neural networks with piecewise-polynomial activation functions. We derive the required depth, width, and sparsity of a deep neural network to approximate any Holder smooth function up to a given approximation error in Holder norms in such a way that all weights of this neural network are bounded by 1. The latter feature is essential to control generalization errors in many statistical and machine learning applications.
Article
Computer Science, Artificial Intelligence
Yunfei Yang, Zhen Li, Yang Wang
Summary: We study the expressive power of deep ReLU neural networks for approximating functions in dilated shift-invariant spaces and estimate the approximation error bounds based on the network's width and depth. Our results have applications in classical function spaces and show that our neural network construction is asymptotically optimal.
Article
Computer Science, Theory & Methods
Jonathan W. Siegel, Jinchao Xu
Summary: This article studies the approximation properties of variation spaces in shallow neural networks with different activation functions. The authors introduce two main tools for estimating the metric entropy, approximation rates, and n-widths of these spaces. They provide upper bounds and lower bounds for the approximation rates, metric entropy, and n-widths of the spaces, which improve upon existing results in many cases.
FOUNDATIONS OF COMPUTATIONAL MATHEMATICS
(2022)
Article
Computer Science, Information Systems
Jonathan W. Siegel, Jinchao Xu
Summary: This article analyzes the situation when the orthogonal greedy algorithm is applied to dictionaries with small entropy. The results show that when the metric entropy of the convex hull decays at a certain rate, the orthogonal greedy algorithm converges at the same rate on the variation space of the dictionary. This result is particularly important for dictionaries corresponding to shallow neural networks.
IEEE TRANSACTIONS ON INFORMATION THEORY
(2022)
Article
Computer Science, Artificial Intelligence
Linmao Yang, Chunhua Wang
Summary: The article proposes an emotional model with variable learning rate and time delay, considering three types of forgetting. By simulating the entire circuit using PSPICE, it provides an option for emotional learning based on memristors.
Article
Operations Research & Management Science
Fouad El Ouardighi, Konstantin Kogan, Giorgio Gnecco, Marcello Sanguineti
ANNALS OF OPERATIONS RESEARCH
(2020)
Article
Computer Science, Artificial Intelligence
Vera Kurkova
NEURAL COMPUTING & APPLICATIONS
(2019)
Article
Computer Science, Hardware & Architecture
Giorgio Gnecco, Yuval Hadas, Marcello Sanguineti
Article
Computer Science, Artificial Intelligence
Vera Kurkova, Marcello Sanguineti
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2019)
Editorial Material
Computer Science, Artificial Intelligence
Lazaros S. Iliadis, Vera Kurkova, Barbara Hammer
NEURAL COMPUTING & APPLICATIONS
(2020)
Article
Transportation
Giorgio Gnecco, Yuval Hadas, Marcello Sanguineti
Summary: This study evaluates the importance of transfer points in public transport networks using the Shapley value from cooperative game theory, developing a special formulation for transfers. Due to the computational requirements for large networks, a Monte Carlo approximation is used. A real-world case study demonstrates the model's viability.
TRANSPORTMETRICA A-TRANSPORT SCIENCE
(2021)
Article
Mathematics
Paul C. Kainen, Vera Kurkova, Andrew Vogt
JOURNAL OF APPROXIMATION THEORY
(2020)
Article
Computer Science, Hardware & Architecture
Mauro Passacantando, Giorgio Gnecco, Yuval Hadas, Marcello Sanguineti
Summary: This study introduces a new framework to investigate Braess' paradox, by utilizing cooperative games with transferable utility to evaluate the contribution of network resources to overall network performance.
Article
Computer Science, Artificial Intelligence
Ksenia Kolykhalova, Giorgio Gnecco, Marcello Sanguineti, Gualtiero Volpe, Antonio Camurri
IEEE TRANSACTIONS ON HUMAN-MACHINE SYSTEMS
(2020)
Article
Computer Science, Artificial Intelligence
Vera Kurkova, Marcello Sanguineti
Summary: This study investigates the classification of large data sets using feedforward neural networks. A probabilistic model of their relevance is considered to handle unmanageably large sets of classification tasks. The optimization of networks computing randomly chosen classifiers is studied in terms of correlations of classifiers with network input-output functions, and the effects of increasing sizes of data sets on classifications are analyzed using geometrical properties of high-dimensional spaces.
Article
Computer Science, Artificial Intelligence
Vera Kurkova, David Coufal
Summary: The study investigates the suitability of shallow networks with translation-invariant kernel units for function approximation and classification tasks. A critical property influencing the capabilities of kernel networks is how the Fourier transforms of kernels converge to zero. Kernels suitable for multivariable approximation can have negative values but must be almost everywhere nonzero, while kernels suitable for maximal margin classification must be nonnegative everywhere but can have large sets where they are zero.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2021)
Proceedings Paper
Computer Science, Artificial Intelligence
Ming Li, Giorgio Gnecco, Marcello Sanguineti
Summary: This work investigates the effective dimension of the output of a one-hidden-layer neural network with random inner weights. A polynomial approximation is used to estimate the input range, and the results show that the Root Mean Square Error (RMSE) is influenced by the effective dimension and the quality of the features associated with the hidden layer's output.
AI 2021: ADVANCES IN ARTIFICIAL INTELLIGENCE
(2022)
Article
Engineering, Biomedical
Erfan Shojaei Barjuei, M. Mahdi Ghazaei Ardakani, Darwin G. Caldwell, Marcello Sanguineti, Jesus Ortiz
IEEE TRANSACTIONS ON MEDICAL ROBOTICS AND BIONICS
(2020)
Proceedings Paper
Computer Science, Artificial Intelligence
Vera Kurkova
ARTIFICIAL NEURAL NETWORKS AND MACHINE LEARNING - ICANN 2019: THEORETICAL NEURAL COMPUTATION, PT I
(2019)
