Article
Public, Environmental & Occupational Health
Ning Liu, Kexue Luo, Zhenming Yuan, Yan Chen
Summary: A transfer learning model based on speech and natural language processing technology was developed for early diagnosis of Alzheimer's disease. The model achieved an accuracy of 0.88, significantly improving AD prediction.
FRONTIERS IN PUBLIC HEALTH
(2022)
Article
Biochemical Research Methods
Taha ValizadehAslani, Yiwen Shi, Ping Ren, Jing Wang, Yi Zhang, Meng Hu, Liang Zhao, Hualou Liang
Summary: Human prescription drug labeling provides essential scientific information for the safe and effective use of drugs. Automatic information extraction from drug labels using NLP techniques, especially BERT, has shown exceptional performance. The development of PharmBERT, a BERT model pretrained specifically on drug labels, has demonstrated superior performance in multiple NLP tasks in the drug label domain.
BRIEFINGS IN BIOINFORMATICS
(2023)
Article
Computer Science, Artificial Intelligence
Flor Miriam Plaza-del-Arco, M. Dolores Molina-Gonzalez, L. Alfonso Urena-Lopez, M. Teresa Martin-Valdivia
Summary: The paper discusses the task of Spanish hate speech identification on social media and the capabilities of new techniques based on machine learning. The study compares the performance of different methods, with the main contribution being the achievement of promising results in Spanish through the application of multilingual and monolingual pre-trained language models.
EXPERT SYSTEMS WITH APPLICATIONS
(2021)
Article
Chemistry, Multidisciplinary
Pilar Lopez-Ubeda, Flor Miriam Plaza-del-Arco, Manuel Carlos Diaz-Galiano, Maria-Teresa Martin-Valdivia
Summary: Anorexia is a mental disorder involving abnormal nutritional intake behaviors, with early identification and appropriate treatment improving recovery speed. Social media use is strongly associated with eating concerns, and Natural Language Processing can aid in early anorexia detection in textual data. Transfer learning techniques, particularly using Transformer-based models, show promise in detecting anorexia in Spanish tweets.
APPLIED SCIENCES-BASEL
(2021)
Article
Computer Science, Artificial Intelligence
Keno K. Bressem, Jens-Michalis Papaioannou, Paul Grundmann, Florian Borchert, Lisa C. Adams, Leonhard Liu, Felix Busch, Lina Xu, Jan P. Loyen, Stefan M. Niehues, Moritz Augustin, Lennart Grosser, Marcus R. Makowski, Hugo J. W. L. Aerts, Alexander Loeser
Summary: This paper presents medBERT.de, a pre-trained German BERT model designed specifically for the German medical domain. The model achieves state-of-the-art performance on various medical benchmarks and the analysis investigates the impact of data deduplication and tokenization methods on the model's performance.
EXPERT SYSTEMS WITH APPLICATIONS
(2024)
Article
Chemistry, Multidisciplinary
Somaiyeh Dehghan, Mehmet Fatih Amasyali
Summary: Semantic Textual Similarity (STS) is an important task in Natural Language Processing (NLP) that measures the similarity of two texts based on their underlying semantics. However, BERT-derived sentence embeddings are not so robust in the STS task as they cannot capture the full semantic meaning of the sentences due to their dependence on word frequency. This paper proposes a new model called SupMPN, which learns the semantic meanings of sentences by contrasting multiple similar and dissimilar sentences.
APPLIED SCIENCES-BASEL
(2022)
Article
Computer Science, Artificial Intelligence
Daniel Fernandez-Gonzalez
Summary: Semantic role labeling (SRL) recognizes the predicate-argument structure of a sentence and is crucial for various natural language processing tasks. Most existing methods for SRL are not comprehensive and rely on pre-identified predicates, often following a pipeline approach. These approaches heavily depend on syntactic information, making them impractical for real-world applications. This article presents a novel transition-based SRL approach that processes input sentences in a single left-to-right pass without leveraging syntactic information or additional modules. The proposed method achieves state-of-the-art performance on the CoNLL-2009 shared task for multiple languages.
KNOWLEDGE-BASED SYSTEMS
(2023)
Article
Computer Science, Artificial Intelligence
Jose Alberto Benitez-Andrades, Alvaro Gonzalez-Jimenez, Alvaro Lopez-Brea, Jose Aveleira-Mata, Jose-Manuel Alija-Perez, Maria Teresa Garcia-Ordas
Summary: With the growth of social networks, manual content moderation is no longer feasible. This study developed a BERT-based model to detect racist and xenophobic messages in Spanish tweets and compared it with other deep learning models. The results showed that the BETO model achieved the best precision, demonstrating the importance of developing native transfer learning models for Spanish NLP problems.
PEERJ COMPUTER SCIENCE
(2022)
Article
Biochemistry & Molecular Biology
Hitoshi Iuchi, Taro Matsutani, Keisuke Yamada, Natsuki Iwano, Shunsuke Sumi, Shion Hosoda, Shitao Zhao, Tsukasa Fukunaga, Michiaki Hamada
Summary: Representation learning in biological sequence analysis is a critical method for converting biological sequences into vectors for function and structure estimation. By treating biological sequences as sentences and applying natural language processing techniques, we can better analyze and utilize large amounts of sequencing data.
COMPUTATIONAL AND STRUCTURAL BIOTECHNOLOGY JOURNAL
(2021)
Article
Acoustics
Hao Fei, Shengqiong Wu, Yafeng Ren, Donghong Ji
Summary: This paper introduces a second-order end-to-end SRL model and a structural refinement mechanism, which significantly outperforms traditional methods, effectively handling data scarcity and long-range dependency issues.
IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING
(2021)
Article
Multidisciplinary Sciences
Diana Hicks, Matteo Zullo, Ameet Doshi, Omar Asensio
Summary: This study aims to explore how the public utilizes high-quality, scientifically based information and provides detailed evidence of the demand for such information and its wide application in service provision. By using the BERT model to classify comments left by US downloaders, the research sheds light on the importance of protecting and providing access to this type of information.
PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA
(2022)
Article
Computer Science, Artificial Intelligence
Tolga Dimlioglu, Jing Wang, Devansh Bisla, Anna Choromanska, Simon Odie, Leon Bukhman, Afolabi D. Olomola, James Wong
Summary: The operations of large utility companies heavily rely on regulation documents to inform them of policy changes and compliance requirements. We propose an automatic document classification pipeline that determines the importance of a document and forwards it to relevant departments. Our pipeline uses an ensemble of classifiers for binary classification and a transformer-based model for multi-label classification, achieving an accuracy score over 80% on a large corpus of text data.
NEURAL COMPUTING & APPLICATIONS
(2023)
Article
Mathematics
Ping Lou, Dan Yu, Xuemei Jiang, Jiwei Hu, Yuhang Zeng, Chuannian Fan
Summary: Under the background of intelligent manufacturing, industrial systems are developing in a more complex and intelligent direction. Equipment maintenance management is facing significant challenges in terms of maintenance workload, system reliability and stability requirements and the overall skill requirements of maintenance personnel. Equipment maintenance management is also developing in the direction of intellectualization.
Article
Multidisciplinary Sciences
Yoojoong Kim, Jong-Ho Kim, Jeong Moon Lee, Moon Joung Jang, Yun Jin Yum, Seongtae Kim, Unsub Shin, Young-Min Kim, Hyung Joon Joo, Sanghoun Song
Summary: This study presents a Korean medical language model based on deep learning NLP, which showed significant improvement in accuracy through the use of pre-training framework in the medical context.
SCIENTIFIC REPORTS
(2022)
Article
Chemistry, Multidisciplinary
Petros Eleftheriadis, Isidoros Perikos, Ioannis Hatzilygeroudis
Summary: Natural language inference (NLI) is a crucial natural language understanding (NLU) task that involves inferring information during spoken or written communication. This paper presents experimental results on using modern deep learning models, including pre-trained transformer BERT and LSTM-based models, for NLI. The study compares different models on multiple NLI datasets, achieves state-of-the-art results, and examines inference ability and generalization power of the models.
APPLIED SCIENCES-BASEL
(2023)
Article
Engineering, Electrical & Electronic
Taha ValizadehAslani, Abolfazl Falahati
PHYSICAL COMMUNICATION
(2018)
Article
Biochemical Research Methods
Taha ValizadehAslani, Yiwen Shi, Ping Ren, Jing Wang, Yi Zhang, Meng Hu, Liang Zhao, Hualou Liang
Summary: Human prescription drug labeling provides essential scientific information for the safe and effective use of drugs. Automatic information extraction from drug labels using NLP techniques, especially BERT, has shown exceptional performance. The development of PharmBERT, a BERT model pretrained specifically on drug labels, has demonstrated superior performance in multiple NLP tasks in the drug label domain.
BRIEFINGS IN BIOINFORMATICS
(2023)
Article
Biology
Taha ValizadehAslani, Zhengqiao Zhao, Bahrad A. Sokhansanj, Gail L. Rosen