Article
Multidisciplinary Sciences
Mitsumasa Nakajima, Katsuma Inoue, Kenji Tanaka, Yasuo Kuniyoshi, Toshikazu Hashimoto, Kohei Nakajima
Summary: The research presents a physical deep learning approach that can train physical neural networks without knowledge of the physical system and its gradient. Through the use of an optoelectronic recurrent neural network, the concept was validated with competitive accelerated computation performance on benchmarks.
NATURE COMMUNICATIONS
(2022)
Article
Multidisciplinary Sciences
Mohamed Kentour, Joan Lu
Summary: The study reveals a good balance between transparency and efficiency can be achieved in deep neural networks by exploring Credit Assignment Paths theory. Experimental results on the Twitter Health News dataset show the model is transparent and traceable, with an overall accuracy of approximately 83% and around 94% correct identification of positive sentiments.
Article
Multidisciplinary Sciences
Logan G. Wright, Tatsuhiro Onodera, Martin M. Stein, Tianyu Wang, Darren T. Schachter, Zoey Hu, Peter L. McMahon
Summary: The study introduced a hybrid in situ-in silico algorithm called physics-aware training, which applies backpropagation to train controllable physical systems for deep physical neural networks. By demonstrating the training of diverse physical neural networks in areas like optics, mechanics, and electronics to perform audio and image classification tasks, the research showcased the universality and effectiveness of the approach.
Article
Optics
Ziyu Gu, Zicheng Huang, Yesheng Gao, Xingzhao Liu
Summary: This study proposes an in-situ training algorithm in optics to reduce computational burden and take full advantage of optical computing. By performing forward propagation and backward propagation on the same optical system, and introducing optical nonlinearity, the feasibility of the proposed algorithm is validated on several datasets.
Article
Computer Science, Artificial Intelligence
Feng Lin
Summary: The article presents a new learning algorithm that is mathematically equivalent to backpropagation algorithm but eliminates the need for a feedback network, making implementation simpler and increasing biological plausibility for biological neural networks to learn using the new algorithm.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2022)
Article
Chemistry, Multidisciplinary
Aleksandra Baczkiewicz, Jaroslaw Watrobski, Wojciech Salabun, Joanna Kolodziejczyk
Summary: The study utilized a regressive model based on a Multilayer Perceptron to predict weather indicators for the city of Szczecin in Poland, achieving a high level of accuracy in predicting minimum and maximum temperatures for the next day and a good performance in predicting atmospheric pressure.
APPLIED SCIENCES-BASEL
(2021)
Review
Computer Science, Artificial Intelligence
Urszula Markowska-Kaczmar, Michal Kosturek
Summary: The research compares randomisation-based learning with classical feedforward neural networks and finds that extreme learning machines perform better on relatively small datasets, but further development is needed for demanding image datasets. Smart algorithms for inverse matrix calculation and specific mechanisms to avoid memory overhead are suggested for ELM improvement.
NEURAL COMPUTING & APPLICATIONS
(2021)
Article
Computer Science, Artificial Intelligence
Zhanglu Yan, Jun Zhou, Weng-Fai Wong
Summary: In this paper, the authors propose a new SNN-compatible training algorithm called CQ+ that achieves state-of-the-art accuracy on CIFAR-10 and CIFAR-100 datasets. By using a parameterized input encoding method and a threshold training method, they were able to reduce the latency and achieve near-zero accuracy loss when transforming popular CNN models to SNNs with a smaller time window size. The framework developed in PyTorch is publicly available.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
(2023)
Article
Computer Science, Hardware & Architecture
Gwangho Lee, Sunwoo Lee, Dongsuk Jeon
Summary: This study introduces a local learning algorithm that significantly reduces computational complexity and improves training performance, while eliminating the drawbacks of the backpropagation algorithm by performing local learning between blocks.
IEEE TRANSACTIONS ON VERY LARGE SCALE INTEGRATION (VLSI) SYSTEMS
(2021)
Article
Multidisciplinary Sciences
Changqing Xu, Yi Liu, Dongdong Chen, Yintang Yang
Summary: A novel training method based on backpropagation for ultra-low-latency spiking neural networks has been proposed in this study, which increases the information capacity of each spike using the multi-threshold Leaky Integrate and Fired (LIF) model. Experimental results demonstrate the method achieves high accuracy on MNIST, FashionMNIST, and CIFAR10 datasets, with significant improvement on CIFAR10 compared to previous approaches.
Article
Multidisciplinary Sciences
Wei Fang, Yanqi Chen, Jianhao Ding, Zhaofei Yu, Timothee Masquelier, Ding Chen, Liwei Huang, Huihui Zhou, Guoqi Li, Yonghong Tian
Summary: Spiking neural networks (SNNs) aim to achieve brain-inspired intelligence on neuromorphic chips with high energy efficiency. However, traditional programming frameworks cannot meet the demands of automatic differentiation, parallel computation acceleration, and deployment. In this study, a framework called SpikingJelly is proposed to address these issues, which can accelerate the training of deep SNNs and provide flexible model acceleration capabilities.
Article
Computer Science, Artificial Intelligence
Takashi Matsubara, Yuto Miyatake, Takaharu Yaguchi
Summary: The combination of neural networks and numerical integration can provide highly accurate models. The symplectic adjoint method proposed in this study obtains the exact gradient with much less memory cost and is more robust to rounding errors than the naive backpropagation algorithm and checkpointing schemes.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2023)
Article
Computer Science, Artificial Intelligence
Saban Gulcu
Summary: The training algorithm is a crucial component of artificial neural networks (ANN) that affects their performance. This article presents a new hybrid algorithm called DA-MLP, which uses the dragonfly algorithm to train feed-forward multilayer neural networks (MLP). The experimental study shows that the DA-MLP algorithm is more efficient than other algorithms.
APPLIED SOFT COMPUTING
(2022)
Article
Engineering, Electrical & Electronic
Nanda K. Unnikrishnan, Keshab K. Parhi
Summary: This paper presents a novel approach to designing accelerators for training convolutional neural networks using systolic architectures. The proposed method interleaves the computation of activation function gradients and weight gradients on the same systolic array, resulting in improved variable reuse and reduced unnecessary memory accesses and energy consumption. Experimental results show significant savings in terms of cycle count, memory accesses, and energy consumption, as compared to baseline implementations for state-of-the-art CNNs.
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS I-REGULAR PAPERS
(2023)
Article
Engineering, Electrical & Electronic
Su-in Yi, Jack D. Kendall, R. Stanley Williams, Suhas Kumar
Summary: This article reports on a training method using activity-difference-based training on co-designed analogue memristor crossbars. It treats network parameters as a constrained optimization problem and numerically calculates local gradients using behavioral differences. The trained neural networks can classify Braille words with high accuracy.
NATURE ELECTRONICS
(2023)
Article
Computer Science, Artificial Intelligence
Hamdan Abdellatef, Lina J. Karam
Summary: This paper proposes performing the learning and inference processes in the compressed domain to reduce computational complexity and improve speed of neural networks. Experimental results show that modified ResNet-50 in the compressed domain is 70% faster than traditional spatial-based ResNet-50 while maintaining similar accuracy. Additionally, a preprocessing step with partial encoding is suggested to improve resilience to distortions caused by low-quality encoded images. Training a network with highly compressed data can achieve good classification accuracy with significantly reduced storage requirements.
Article
Computer Science, Artificial Intelligence
Victor R. Barradas, Yasuharu Koike, Nicolas Schweighofer
Summary: Inverse models are essential for human motor learning as they map desired actions to motor commands. The shape of the error surface and the distribution of targets in a task play a crucial role in determining the speed of learning.
Article
Computer Science, Artificial Intelligence
Ting Zhou, Hanshu Yan, Jingfeng Zhang, Lei Liu, Bo Han
Summary: We propose a defense strategy that reduces the success rate of data poisoning attacks in downstream tasks by pre-training a robust foundation model.
Article
Computer Science, Artificial Intelligence
Hao Sun, Li Shen, Qihuang Zhong, Liang Ding, Shixiang Chen, Jingwei Sun, Jing Li, Guangzhong Sun, Dacheng Tao
Summary: In this paper, the convergence rate of AdaSAM in the stochastic non-convex setting is analyzed. Theoretical proof shows that AdaSAM has a linear speedup property and decouples the stochastic gradient steps with the adaptive learning rate and perturbed gradient. Experimental results demonstrate that AdaSAM outperforms other optimizers in terms of performance.
Article
Computer Science, Artificial Intelligence
Juntong Yun, Du Jiang, Li Huang, Bo Tao, Shangchun Liao, Ying Liu, Xin Liu, Gongfa Li, Disi Chen, Baojia Chen
Summary: In this study, a dual manipulator grasping detection model based on the Markov decision process is proposed. By parameterizing the grasping detection model of dual manipulators using a cross entropy convolutional neural network and a full convolutional neural network, stable grasping of complex multiple objects is achieved. Robot grasping experiments were conducted to verify the feasibility and superiority of this method.
Article
Computer Science, Artificial Intelligence
Miaohui Zhang, Kaifang Li, Jianxin Ma, Xile Wang
Summary: This paper proposes an unsupervised person re-identification (Re-ID) method that uses two asymmetric networks to generate pseudo-labels for each other by clustering and updates and optimizes the pseudo-labels through alternate training. It also designs similarity compensation and similarity suppression based on the camera ID of pedestrian images to optimize the similarity measure. Extensive experiments show that the proposed method achieves superior performance compared to state-of-the-art unsupervised person re-identification methods.
Article
Computer Science, Artificial Intelligence
Florian Bacho, Dominique Chu
Summary: This paper proposes a new approach called the Forward Direct Feedback Alignment algorithm for supervised learning in deep neural networks. By combining activity-perturbed forward gradients, direct feedback alignment, and momentum, this method achieves better performance and convergence speed compared to other local alternatives to backpropagation.
Article
Computer Science, Artificial Intelligence
Xiaojian Ding, Yi Li, Shilin Chen
Summary: This research paper addresses the limitations of recursive feature elimination (RFE) and its variants in high-dimensional feature selection tasks. The proposed algorithms, which introduce a novel feature ranking criterion and an optimal feature subset evaluation algorithm, outperform current state-of-the-art methods.
Article
Computer Science, Artificial Intelligence
Naoko Koide-Majima, Shinji Nishimoto, Kei Majima
Summary: Visual images observed by humans can be reconstructed from brain activity, and the visualization of arbitrary natural images from mental imagery has been achieved through an improved method. This study provides a unique tool for directly investigating the subjective contents of the brain.
Article
Computer Science, Artificial Intelligence
Huanjie Tao, Qianyue Duan
Summary: In this paper, a hierarchical attention network with progressive feature fusion is proposed for facial expression recognition (FER), addressing the challenges posed by pose variation, occlusions, and illumination variation. The model achieves enhanced performance by aggregating diverse features and progressively enhancing discriminative features.
Article
Computer Science, Artificial Intelligence
Zhenyi Wang, Pengfei Yang, Linwei Hu, Bowen Zhang, Chengmin Lin, Wenkai Lv, Quan Wang
Summary: In the face of the complex landscape of deep learning, we propose a novel subgraph-level performance prediction method called SLAPP, which combines graph and operator features through an innovative graph neural network called EAGAT, providing accurate performance predictions. In addition, we introduce a mixed loss design with dynamic weight adjustment to improve predictive accuracy.
Article
Computer Science, Artificial Intelligence
Yiyang Yin, Shuangling Luo, Jun Zhou, Liang Kang, Calvin Yu-Chian Chen
Summary: Medical image segmentation is crucial for modern healthcare systems, especially in reducing surgical risks and planning treatments. Transanal total mesorectal excision (TaTME) has become an important method for treating colon and rectum cancers. Real-time instance segmentation during TaTME surgeries can assist surgeons in minimizing risks. However, the dynamic variations in TaTME images pose challenges for accurate instance segmentation.
Article
Computer Science, Artificial Intelligence
Teng Cheng, Lei Sun, Junning Zhang, Jinling Wang, Zhanyang Wei
Summary: This study proposes a scheme that combines the start-stop point signal features for wideband multi-signal detection, called Fast Spectrum-Size Self-Training network (FSSNet). By utilizing start-stop points to build the signal model, this method successfully solves the difficulty of existing deep learning methods in detecting discontinuous signals and achieves satisfactory detection speed.
Article
Computer Science, Artificial Intelligence
Wenming Wu, Xiaoke Ma, Quan Wang, Maoguo Gong, Quanxue Gao
Summary: The layer-specific modules in multi-layer networks are critical for understanding the structure and function of the system. However, existing methods fail to accurately characterize and balance the connectivity and specificity of these modules. To address this issue, a joint learning graph clustering algorithm (DRDF) is proposed, which learns the deep representation and discriminative features of the multi-layer network, and balances the connectivity and specificity of the layer-specific modules through joint learning.
Article
Computer Science, Artificial Intelligence
Guanghui Yue, Guibin Zhuo, Weiqing Yan, Tianwei Zhou, Chang Tang, Peng Yang, Tianfu Wang
Summary: This paper proposes a novel boundary uncertainty aware network (BUNet) for precise and robust colorectal polyp segmentation. BUNet utilizes a pyramid vision transformer encoder to learn multi-scale features and incorporates a boundary exploration module (BEM) and a boundary uncertainty aware module (BUM) to handle boundary areas. Experimental results demonstrate that BUNet outperforms other methods in terms of performance and generalization ability.
