Article
Engineering, Electrical & Electronic
Bi Wu, Zhengkuan Wang, Ke Chen, Chenggang Yan, Weiqiang Liu
Summary: This paper investigates the Gating Units Level Balanced Compression (GBC) strategy in recurrent neural networks (RNNs), achieving high compression rates for long short-term memory (LSTM) models while reducing additional parameters. Experimental results demonstrate that GBC significantly reduces storage space and computational complexity of LSTM models while maintaining accuracy, and hardware experiments show improved energy efficiency compared to state-of-the-art designs.
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS I-REGULAR PAPERS
(2022)
Article
Engineering, Civil
Yujia Zhai, Yan Wan, Xiaoxiao Wang
Summary: The bidirectional long and short-term memory (BiLSTM) algorithm is used for traffic flow prediction, and the results show that it has a relatively high prediction accuracy, outperforming the traditional long short-term memory(LSTM) algorithm in fitting the actual situation. The prediction results of BiLSTM are consistent with the actual situation during the stationary period and the low peak period, and slightly different during the peak period, but still have reference value.
JOURNAL OF ADVANCED TRANSPORTATION
(2022)
Article
Engineering, Electrical & Electronic
Guocai Nan, Zhengkuan Wang, Chenghua Wang, Bi Wu, Zhican Wang, Weiqiang Liu, Fabrizio Lombardi
Summary: This work introduces a hybrid-iterative compression algorithm for LSTM/GRU and proposes an energy-efficient accelerator for bidirectional RNNs. By grouping gating units and using different compression algorithms, significant reduction in storage and computation requirements can be achieved without compromising accuracy. Improvements in the data flow of matrix operation unit and BRAM utilization, along with a timing matching strategy, address the load-imbalance issue and result in enhanced energy efficiency compared to state-of-the-art designs.
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS I-REGULAR PAPERS
(2021)
Article
Computer Science, Artificial Intelligence
Philip B. Weerakody, Kok Wai Wong, Guanjin Wang
Summary: Cyclic Gate enhanced recurrent neural networks with learnt waveform parameters are developed to automatically identify important data segments within a time series and neglect unimportant segments. The results of testing on multiple time series datasets show that this method can improve model performance in the presence of missing values.
NEURAL PROCESSING LETTERS
(2023)
Article
Infectious Diseases
Albert Bolatchiev, Vladimir Baturin, Evgeny Shchetinin, Elizaveta Bolatchieva
Summary: The search for new antibiotics is important in the face of widespread antibiotic resistance. One promising strategy is the de novo design of novel antimicrobial peptides. By using a neural network, the amino acid sequences of 198 novel peptides were obtained, and two of them showed effective antimicrobial activity.
Article
Computer Science, Information Systems
Majid Mosavat, Guido Montorsi
Summary: In this article, we explore the feasibility of Terrestrial Broadcasting using a Recurrent Neural Network (RNN) as the receiver in a Single-Frequency Network (SFN) with 5G numerology. Compared to traditional equalization techniques, the RNN receiver proves to be more effective in handling channel estimation and equalization, especially in scenarios with interference. Through parameter optimization and comparison with a classical OFDM system, we demonstrate the superiority of our proposed system and the robustness of the RNN receiver.
Article
Engineering, Electrical & Electronic
Mustafa Mert Keskin, Fatih Irim, Oguzhan Karaahmetoglu, Ersin Kaya
Summary: This paper investigates the modeling capability of LSTM for distant temporal interaction and proposes a novel hierarchical architecture (HLSTM) to enhance this capability. Experimental results demonstrate that the new architecture outperforms the traditional LSTM architecture and other studies in modeling deep temporal connections.
SIGNAL IMAGE AND VIDEO PROCESSING
(2023)
Article
Computer Science, Interdisciplinary Applications
Shrouq Alelaumi, Nourma Khader, Jingxi He, Sarah Lam, Sang Won Yoon
Summary: This research proposes a novel framework for controlling the selection of the cleaning profile in the stencil printing process. By using a multi-dimensional temporal recurrent neural network (RNN) approach, the amount of residue buildup on the stencil's surface can be accurately predicted in real-time. Experimental results demonstrate that the proposed LSTM model outperforms existing regression models in predicting stencil status accurately.
ROBOTICS AND COMPUTER-INTEGRATED MANUFACTURING
(2021)
Article
Biophysics
B. Puszkarski, K. Hryniow, G. Sarwas
Summary: This study analyzes the ability of the N-BEATS architecture for the prediction and classification of ECG signals, comparing its performance with other commonly used recurrent neural network architectures. The results show that while N-BEATS may not perform as well as LSTM in terms of multi-label classification and dataset resilience, it outperforms other solutions in terms of complexity and speed.
PHYSIOLOGICAL MEASUREMENT
(2022)
Article
Genetics & Heredity
Wenjie Cao, Ya-Zhou Shi, Huahai Qiu, Bengong Zhang
Summary: This study introduces a new algorithm named SS-RNN that uses multiple historical information to predict the current time information, addressing the issues in long-term memory and gradient back propagation in recurrent neural networks. By providing different processing methods, it thoroughly explores the impact of historical information on RNNs.
FRONTIERS IN GENETICS
(2021)
Article
Computer Science, Information Systems
Ali Pourranjbar, Georges Kaddoum, Walid Saad
Summary: Conventional anti-jamming methods are ineffective against a single jammer following multiple different jamming policies or multiple jammers with distinct policies. This article proposes an anti-jamming method that can adapt to the current jamming attack and estimates future occupied channels in the multiple jammers scenario. The proposed methods outperform the baseline method and achieve high success rates and ergodic rates.
IEEE INTERNET OF THINGS JOURNAL
(2023)
Article
Computer Science, Artificial Intelligence
Hyungjin Park, Geonseok Lee, Kichun Lee
Summary: This paper proposes a new method for analyzing multivariate time series by connecting features using partial linear dependence to reflect their dependencies. Experimental results show that the proposed method outperforms existing models on multivariate datasets.
EXPERT SYSTEMS WITH APPLICATIONS
(2022)
Article
Computer Science, Information Systems
Shiguang Liu, Huixin Wang, Xiaoli Zhang
Summary: This study presents a video decolorization algorithm based on convolutional neural network and long short-term memory neural network, which preserves the contrast of video frames by learning and extracting the same local content of continuous video frames, and maintains temporal consistency of the video sequence using a temporal feature controller. The proposed method outperforms existing techniques in preserving the local contrast of video frames and temporal consistency.
ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS
(2021)
Article
Chemistry, Multidisciplinary
Alejandro Toro-Ossaba, Juan Jaramillo-Tigreros, Juan C. Tejada, Alejandro Pena, Alexandro Lopez-Gonzalez, Rui Alexandre Castanho
Summary: Research on gesture recognition systems has gained popularity in the field of human-machine interaction. However, the use of extensive channels and electrodes in prosthesis and orthesis for gesture recognition increases complexity and cost. This paper presents a gesture classifier based on a Recurrent Neural Network (RNN) model using LSTM units and dense layers to reduce the number of channels and overall complexity, improving scalability for embedded systems. Experimental results show high accuracy in gesture recognition using the proposed model.
APPLIED SCIENCES-BASEL
(2022)
Article
Computer Science, Artificial Intelligence
Zhenyu Lei, Shangce Gao, Hideyuki Hasegawa, Zhiming Zhang, MengChu Zhou, Khaled Sedraoui
Summary: Ultrasound imaging is widely used in medical diagnosis due to its real-time capability, cost-efficiency, noninvasiveness, and nonionizing nature. However, traditional beamformers have limitations in resolution and contrast. Adaptive beamformers have been proposed to improve image quality, but they are computationally expensive. This study introduces a fully complex-valued gated recurrent neural network for training an ultrasound imaging model that enhances image quality.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2023)
Proceedings Paper
Acoustics
Mohammad Zeineldeen, Jingjing Xu, Christoph Luescher, Wilfried Michel, Alexander Gerstenberger, Ralf Schlueter, Hermann Ney
Summary: The paper investigates the application of a Conformer-based hybrid model in ASR, exploring different training aspects and methods to improve performance. The results show competitive performance and significant improvement compared to other architectures.
2022 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP)
(2022)
Proceedings Paper
Acoustics
Wei Zhou, Zuoyun Zheng, Ralf Schlueter, Hermann Ney
Summary: This paper studies the LM integration methods based on ILM correction in the RNN-T framework. A decoding interpretation is provided, and two reasons for performance improvement with ILM correction are experimentally verified. The proposed exact-ILM training framework is also introduced to theoretically justify other ILM approaches.
2022 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP)
(2022)
Proceedings Paper
Acoustics
Tina Raissi, Eugen Beck, Ralf Schlueter, Hermann Ney
Summary: This work introduces a factored hybrid hidden Markov model (FH-HMM) that outperforms state-of-the-art hybrid HMM without phonetic state-tying. The FH-HMM links to transducer models in how it models phonetic context while maintaining the separation of acoustic and language model components. It can be trained from scratch without using phonetic state-tying and enables triphone context while avoiding phonetic state-tying.
2022 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP)
(2022)
Proceedings Paper
Acoustics
Nils-Philipp Wynands, Wilfried Michel, Jan Rosendahl, Ralf Schluter, Hermann Ney
Summary: Sequence discriminative training is an important tool for improving the performance of automatic speech recognition systems. However, the computation of sum over all possible word sequences is impractical. Current state-of-the-art systems overcome this problem by limiting the summation to a selected number of relevant hypotheses obtained from beam search. This study proposes an approximate method of hypothesis recombination during beam search, which allows for a significant increase in the effective beam size without increasing computational requirements.
2022 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP)
(2022)
Proceedings Paper
Engineering, Biomedical
Michael Gansen, Jie Lou, Florian Freye, Tobias Gemmeke, Farhad Merchant, Albert Zeyer, Mohammad Zeineldeen, Ralf Schlueter, Xin Fan
Summary: This paper presents recent studies on digital approximate computing, exploring discrete approximation using floating-point number representations and addressing time-domain computing. The proposed approximate arithmetic and nonlinear activation functions achieve competitive Quality-of-Service compared to full-precision computing in various artificial neural networks.
PROCEEDINGS OF THE TWENTY THIRD INTERNATIONAL SYMPOSIUM ON QUALITY ELECTRONIC DESIGN (ISQED 2022)
(2022)
Proceedings Paper
Audiology & Speech-Language Pathology
Mohammad Zeineldeen, Aleksandr Glushko, Wilfried Michel, Albert Zeyer, Ralf Schlueter, Hermann Ney
Summary: This study focuses on the implicit internal language model (ILM) within attention-based encoder-decoder models, proposing various methods to estimate ILM directly and surpassing previous approaches. Additionally, other methods to suppress ILM are explored, such as reducing model capacity, limiting label context, and training the model with an existing LM simultaneously.
Proceedings Paper
Audiology & Speech-Language Pathology
Albert Zeyer, Andre Merboldt, Wilfried Michel, Ralf Schluter, Hermann Ney
Summary: Our study on the transducer model suggests that subtracting the estimated internal LM can lead to over 14% relative improvement over normal shallow fusion. The model has a separate probability distribution for non-blank labels, making it easier to combine with external LM and estimate the internal LM. Additionally, the inclusion of the end-of-sentence (EOS) probability of the external LM in the last blank probability further improves performance.
Proceedings Paper
Audiology & Speech-Language Pathology
Wei Zhou, Mohammad Zeineldeen, Zuoyun Zheng, Ralf Schlueter, Hermann Ney
Summary: This paper introduces an acoustic data-driven subword modeling approach that produces labels suitable for various ASR models. Experimental results demonstrate that this approach outperforms traditional BPE and PASM methods in terms of performance.
Proceedings Paper
Audiology & Speech-Language Pathology
Wei Zhou, Albert Zeyer, Andre Merboldt, Ralf Schlueter, Hermann Ney
Summary: With the introduction of direct models in automatic speech recognition, the traditional frame-wise acoustic modeling based on hidden Markov models has diversified into various architectures. This work proves the equivalence of RNN-Transducer models and segmental models, providing initial experiments on decoding and beam-pruning using the same underlying model.
Proceedings Paper
Audiology & Speech-Language Pathology
Yu Qiao, Wei Zhou, Elma Kerz, Ralf Schlueter
Summary: This study focuses on the impact of using state-of-the-art ASR system on subsequent automatic analysis of linguistic complexity in spontaneously produced L2 speech. Through correlation analysis and controlling for task type effects, a more differential effect of ASR performance on specific types of complexity measures is presented.
Proceedings Paper
Audiology & Speech-Language Pathology
Yingbo Gao, David Thulke, Alexander Gerstenberger, Khoa Viet Tran, Ralf Schlueter, Hermann Ney
Summary: With the increasing vocabulary size of language models, many sampling-based training criteria have been proposed and investigated, simplifying softmax-related traversal over the entire vocabulary for speedups. Contrary to common belief, experimental results show that all these sampling methods can perform equally well when correcting for intended class posterior probabilities.
Proceedings Paper
Computer Science, Artificial Intelligence
Nick Rossenbach, Mohammad Zeineldeen, Benedikt Hilmes, Ralf Schlueter, Hermann Ney
Summary: Recent publications in automatic speech recognition focus on attention encoder-decoder architectures and the use of synthetic data generated by TTS systems. The effectiveness of synthesized data for AED-ASR training is influenced by various factors such as pre-processing, speaker embedding, and internal language model subtraction. Hybrid ASR systems outperform AED systems on LibriSpeech-100h, achieving a lower word error rate on clean/noisy test sets.
2021 IEEE AUTOMATIC SPEECH RECOGNITION AND UNDERSTANDING WORKSHOP (ASRU)
(2021)
Proceedings Paper
Computer Science, Artificial Intelligence
Peter Vieting, Christoph Luescher, Wilfried Michel, Ralf Schlueter, Hermann Ney
Summary: This study investigates acoustic modeling and feature extractors learning, explores the usefulness of unsupervised pre-training feature extractors in ASR systems, compares the performance of different feature sets, and discusses how to further improve the system performance.
2021 IEEE AUTOMATIC SPEECH RECOGNITION AND UNDERSTANDING WORKSHOP (ASRU)
(2021)
Proceedings Paper
Acoustics
Wei Zhou, Simon Berger, Ralf Schlueter, Hermann Ney
Summary: This study presents a phoneme-based neural transducer modeling approach that combines the advantages of classical and end-to-end methods by improving alignment label topologies, enhancing phoneme labels, and utilizing local phonetic dependencies along with external language models for sequence-to-sequence modeling consistency. The training procedure using frame-wise cross-entropy loss and a phonetic context size of one are shown to be efficient for achieving optimal performance.
2021 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP 2021)
(2021)
Proceedings Paper
Computer Science, Artificial Intelligence
Parnia Bahar, Tobias Bieschke, Ralf Schlueter, Hermann Ney
Summary: This study explores the feasibility of collapsing cascaded speech translation models into a single end-to-end trainable model by optimizing all parameters of ASR and MT models jointly. Experimental results show that the model outperforms cascade models and direct models, achieving higher performance in terms of BLEU and TER.
2021 IEEE SPOKEN LANGUAGE TECHNOLOGY WORKSHOP (SLT)
(2021)