Article
Chemistry, Multidisciplinary
Hyeongu Yun, Taegwan Kang, Kyomin Jung
Summary: This paper quantitatively analyzes the inter-head diversity of multi-head attention and proposes a hypothesis that controlling the inter-head diversity can improve model performance. The empirical results show that controlling inter-head diversity leads to better performance compared to baselines.
APPLIED SCIENCES-BASEL
(2021)
Article
Computer Science, Artificial Intelligence
Le Qi, Yu Zhang, Qingyu Yin, Ting Liu
Summary: This paper introduces a Multiple Structural Priors Guided Transformer (MS-Transformer) that integrates different types of structural priors into Transformers using a novel multi-head attention mechanism, aiming to capture the rich structural information of texts. Experimental results demonstrate significant performance improvements of MS-Transformer compared to other models.
COMPUTER SPEECH AND LANGUAGE
(2022)
Article
Computer Science, Artificial Intelligence
Zhe Zhang, Guangli Xiao, Yurong Qian, Mengnan Ma, Hongyong Leng, Tao Zhang
Summary: This study proposes a novel sentence matching model (VSCA) that utilizes a variational autoencoder (VAE)-based attention mechanism to construct basic attention feature maps and combines spatial attention mechanism with visual perception to capture multilevel semantic information. Experimental results demonstrate that VSCA outperforms pretrained models like BERT on the LCQMC dataset and performs well on the PAWS-X data. In addition, VSCA has lower time and space complexity while capturing rich attentional information and detailed information.
COGNITIVE COMPUTATION
(2023)
Article
Psychology
Garrett Smith, Julie Franck, Whitney Tabor
Summary: According to cue-based retrieval theories of sentence comprehension, the syntactic dependency between a verb and the subject is susceptible to interference from other noun phrases in the sentence; a self-organized sentence processing model provides a more parsimonious explanation on encoding interference effects; self-organization approach reduces parsing to feature match optimization, offering a unified method for similarity-based interference in sentence comprehension.
COGNITIVE PSYCHOLOGY
(2021)
Article
Mathematics
Laith H. Baniata, Sangwoo Kang, Isaac K. E. Ampomah
Summary: This paper presents a new reverse positional encoding mechanism for neural machine translation, which effectively handles the open grammatical structure issue of Arabic dialect sentences and enhances the translation quality for right-to-left texts.
Article
Computer Science, Artificial Intelligence
Zuchao Li, Zhuosheng Zhang, Hai Zhao, Rui Wang, Kehai Chen, Masao Utiyama, Eiichiro Sumita
Summary: This paper proposes explicit and implicit text compression approaches to enhance Transformer encoding and improve performance on downstream tasks by integrating backbone information into the Transformer-based models.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
(2022)
Article
Behavioral Sciences
R. Muralikrishnan, Ali Idrissi
Summary: The study demonstrates the importance of subject-verb agreement in language comprehension, with different linguistic features potentially modulating comprehension processes to varying degrees. Violations of subject-verb agreement induce specific ERP effects, which are systematically graded based on the features violated.
Article
Computer Science, Information Systems
Youren Yu, Yangsen Zhang, Xueyang Liu, Siwen Zhu
Summary: This paper proposes a novel relationship extraction model that effectively utilizes interaction information between subjects and objects and captures spatial location relationships between entities. Experimental results show that the model outperforms state-of-the-art models on multiple datasets.
Article
Computer Science, Artificial Intelligence
Chaoming Liu, Wenhao Zhu, Xiaoyu Zhang, Qiuhong Zhai
Summary: In this study, a sentence part-enhanced BERT (SpeBERT) model is proposed, which enhances sentence representations by considering sentence parts with respect to downstream tasks. The sentence parts are encoded based on dependency parsing and downstream tasks, and embeddings are extracted through a pooling operation. Experimental results show that the proposed SpeBERT model outperforms competitor models in sentiment classification and semantic textual similarity tasks.
COMPLEX & INTELLIGENT SYSTEMS
(2023)
Article
Computer Science, Artificial Intelligence
Thi N. N. Dinh, Phu Pham, Giang L. L. Nguyen, Bay Vo
Summary: The process of finding suitable papers related to an interesting topic in research can be time-consuming and labor-intensive. Citation recommendation, using deep learning techniques, has significantly improved the performance of recommending relevant papers. However, existing models still face limitations in integrating auxiliary information for contextual learning. In this paper, we propose a novel context-aware Neural Citation Network (NCN) model with additional textual data integration and BERT model to enhance the performance of citation recommendation. Extensive experiments on the arXiv dataset demonstrate the effectiveness of our proposed model.
APPLIED INTELLIGENCE
(2023)
Article
Psychology, Multidisciplinary
Tao Zeng, Chen Chen, Jiashu Guo
Summary: This study investigates the role of first language translation in second language word processing, finding that it influences bilinguals' task performances. Higher proficiency bilinguals perform better and have better access to first language translation in second language word processing. The depth of first language translation involvement varies based on task demands.
FRONTIERS IN PSYCHOLOGY
(2022)
Article
Psychology
Michael Hahn, Judith Degen, Richard Futrell
Summary: The paper introduces the Efficient Trade-off Hypothesis, which suggests that the order of elements in natural language is influenced by memory and surprisal trade-offs. Through large-scale studies, it is shown that principles of order in language can be explained through efficient trade-offs.
PSYCHOLOGICAL REVIEW
(2021)
Article
Chemistry, Multidisciplinary
Huey-Ing Liu, Wei-Lin Chen
Summary: This study presents a new machine translation model called X-Transformer, which improves the original Transformer model in terms of parameter compression, encoder structure modification, and decoder model size reduction. The X-Transformer achieves better translation results and significantly reduces training time compared to the Transformer model.
APPLIED SCIENCES-BASEL
(2022)
Article
Computer Science, Artificial Intelligence
Myeongjun Jang, Pilsung Kang
Summary: Sentence embedding is a significant research topic in natural language processing (NLP), aiming to generate vectors that represent the intrinsic meaning of sentences and improve performance in NLP tasks. Various approaches have been proposed and evaluated using semantic textual similarity (STS) tasks, with supervised neural network-based models delivering state-of-the-art performance. However, these models have limitations in terms of learnable parameters and the required amount of labeled training data. Pretrained language model-based approaches have emerged as a dominant trend, but acquiring sufficient labeled data is still necessary for fine-tuning.
COMPUTER SPEECH AND LANGUAGE
(2022)
Article
Computer Science, Information Systems
Xue-Liang Leng, Xiao-Ai Miao, Tao Liu
Summary: This paper introduces a new model that combines bidirectional RNN and Enhanced Multi-Head Self-Attention mechanism for sentiment analysis on movie reviews. Experimental results show that the model performs better in terms of accuracy, precision, recall rate, and F1-scores, with BiLSTM outperforming biGRU in the model.
MULTIMEDIA TOOLS AND APPLICATIONS
(2021)
Article
Computer Science, Artificial Intelligence
Rui Lv, Dingheng Wang, Jiangbin Zheng, Zhao-Xu Yang
Summary: In this paper, the authors investigate tensor decomposition for neural network compression. They analyze the convergence and precision of tensor mapping theory, validate the rationality of tensor mapping and its superiority over traditional tensor approximation based on the Lottery Ticket Hypothesis. They propose an efficient method called 3D-KCPNet to compress 3D convolutional neural networks using the Kronecker canonical polyadic (KCP) tensor decomposition. Experimental results show that 3D-KCPNet achieves higher accuracy compared to the original baseline model and the corresponding tensor approximation model.
Article
Computer Science, Artificial Intelligence
Xiangkun He, Zhongxu Hu, Haohan Yang, Chen Lv
Summary: In this paper, a novel constrained multi-objective reinforcement learning algorithm is proposed for personalized end-to-end robotic control with continuous actions. The approach trains a single model using constraint design and a comprehensive index to achieve optimal policies based on user-specified preferences.
Article
Computer Science, Artificial Intelligence
Zhijian Zhuo, Bilian Chen, Shenbao Yu, Langcai Cao
Summary: In this paper, a novel method called Expansion with Contraction Method for Overlapping Community Detection (ECOCD) is proposed, which utilizes non-negative matrix factorization to obtain disjoint communities and applies expansion and contraction processes to adjust the degree of overlap. ECOCD is applicable to various networks with different properties and achieves high-quality overlapping community detection.
Article
Computer Science, Artificial Intelligence
Yizhe Zhu, Chunhui Zhang, Jialin Gao, Xin Sun, Zihan Rui, Xi Zhou
Summary: In this work, the authors propose a Contrastive Spatio-Temporal Distilling (CSTD) approach to improve the detection of high-compressed deepfake videos. The approach leverages spatial-frequency cues and temporal-contrastive alignment to fully exploit spatiotemporal inconsistency information.
Review
Computer Science, Artificial Intelligence
Laijin Meng, Xinghao Jiang, Tanfeng Sun
Summary: This paper provides a review of coverless steganographic algorithms, including the development process, known contributions, and general issues in image and video algorithms. It also discusses the security of coverless steganography from theoretical analysis to actual investigation for the first time.
Article
Computer Science, Artificial Intelligence
Yajie Bao, Tianwei Xing, Xun Chen
Summary: Visual question answering requires processing multi-modal information and effective reasoning. Neural-symbolic learning is a promising method, but current approaches lack uncertainty handling and can only provide a single answer. To address this, we propose a confidence based neural-symbolic approach that evaluates NN inferences and conducts reasoning based on confidence.
Article
Computer Science, Artificial Intelligence
Anh H. Vo, Bao T. Nguyen
Summary: Interior style classification is an interesting problem with potential applications in both commercial and academic domains. This project proposes a method named ISC-DeIT, which combines data-efficient image transformer architectures and knowledge distillation, to address the interior style classification problem. Experimental results demonstrate a significant improvement in predictive accuracy compared to other state-of-the-art methods.
Article
Computer Science, Artificial Intelligence
Shashank Kotyan, Danilo Vasconcellos Vargas
Summary: This article introduces a novel augmentation technique called Dynamic Scanning Augmentation to improve the accuracy and robustness of Vision Transformer (ViT). The technique leverages dynamic input sequences to adaptively focus on different patches, resulting in significant changes in ViT's attention mechanism. Experimental results demonstrate that Dynamic Scanning Augmentation outperforms ViT in terms of both robustness to adversarial attacks and accuracy against natural images.
Article
Computer Science, Artificial Intelligence
Hiba Alqasir, Damien Muselet, Christophe Ducottet
Summary: The article proposes a solution to improve the learning process of a classification network by providing shape priors, reducing the need for annotated data. The solution is tested on cross-domain digit classification tasks and a video surveillance application.
Article
Computer Science, Artificial Intelligence
Dexiu Ma, Mei Liu, Mingsheng Shang
Summary: This paper proposes a method using neural dynamics solvers to solve infinity-norm optimization problems. Two improved solvers are constructed and their effectiveness and superiority are demonstrated through theoretical analysis and simulation experiments.
Article
Computer Science, Artificial Intelligence
Francesco Gregoretti, Giovanni Pezzulo, Domenico Maisto
Summary: Active Inference is a computational framework that uses probabilistic inference and variational free energy minimization to describe perception, planning, and action. cpp-AIF is a header-only C++ library that provides a powerful tool for implementing Active Inference for Partially Observable Markov Decision Processes through multi-core computing. It is cross-platform and improves performance, memory management, and usability compared to existing software.
Article
Computer Science, Artificial Intelligence
Zelin Ying, Dawei Cheng, Cen Chen, Xiang Li, Peng Zhu, Yifeng Luo, Yuqi Liang
Summary: This paper proposes a novel stock market trends prediction framework called SMART, which includes a self-supervised stock technical data sequence embedding model S3E. By training with multiple self-supervised auxiliary tasks, the model encodes stock technical data sequences into embeddings and uses the learned sequence embeddings for predicting stock market trends. Extensive experiments on China A-Shares market and NASDAQ market prove the high effectiveness of our model in stock market trends prediction, and its effectiveness is further validated in real-world applications in a leading financial service provider in China.
Article
Computer Science, Artificial Intelligence
Hao Li, Hao Jiang, Dongsheng Ye, Qiang Wang, Liang Du, Yuanyuan Zeng, Liu Yuan, Yingxue Wang, C. Chen
Summary: DHGAT1, a dynamic hyperbolic graph attention network, utilizes hyperbolic metric properties to embed dynamic graphs. It employs a spatiotemporal self-attention mechanism and weighted node representations, resulting in excellent performance in link prediction tasks.
Article
Computer Science, Artificial Intelligence
Jiehui Huang, Zhenchao Tang, Xuedong He, Jun Zhou, Defeng Zhou, Calvin Yu-Chian Chen
Summary: This study proposes a progressive learning multi-scale feature blending model for image deraining tasks. The model utilizes detail dilation and texture extraction to improve the restoration of rainy images. Experimental results show that the model achieves near state-of-the-art performance in rain removal tasks and exhibits better rain removal realism.
Article
Computer Science, Artificial Intelligence
Lizhi Liu, Zilin Gao, Yinhe Wang, Yongfu Li
Summary: This paper proposes a novel discrete-time interconnected model for depicting complex dynamical networks. The model consists of nodes and edges subsystems, which consider the dynamic characteristic of both nodes and edges. By designing control strategies and coupling modes, the stabilization and synchronization of the network are achieved. Simulation results demonstrate the effectiveness of the proposed methods.