Article
Chemistry, Analytical
Sangyong Park, Jaeseon Kim, Yong Seok Heo
Summary: This study proposes a new regularization method called PALS, which uses self-knowledge distillation to train semantic segmentation networks with limited training data. The method utilizes internal statistics of pixels to generate pixel-wise aggregated probability distributions for increased accuracy. Experimental results show that compared to previous methods, this approach achieves more accurate results with limited training data.
Article
Computer Science, Information Systems
Jingru Li, Sheng Zhou, Liangcheng Li, Haishuai Wang, Jiajun Bu, Zhi Yu
Summary: DFKD is a widely-used strategy for Knowledge Distillation (KD). CuDFKD is a novel DFKD method that utilizes a dynamic strategy and adaptive generation target to improve training effectiveness and stability.
INFORMATION SCIENCES
(2023)
Article
Computer Science, Artificial Intelligence
Yanwu Yang, Xutao Guo, Chenfei Ye, Yang Xiang, Ting Ma
Summary: One of the core challenges of deep learning in medical image analysis is data insufficiency. This paper proposes a Confidence Regularized Knowledge Distillation (CReg-KD) framework to mitigate the issue by penalizing attentive output distributions and intermediate representations based on knowledge confidence. Experimental results demonstrate that CReg-KD outperforms other state-of-the-art knowledge distillation approaches, making it a powerful tool in medical image analysis.
MEDICAL IMAGE ANALYSIS
(2023)
Article
Computer Science, Artificial Intelligence
Hyoje Lee, Yeachan Park, Hyun Seo, Myungjoo Kang
Summary: In order to enhance the performance of deep neural networks, a self-knowledge distillation method called SD-Dropout is proposed in this paper. It distills the posterior distributions of multiple models through dropout sampling, without the need for additional trainable modules or reliance on data. This simple method improves the generalization of a single network and also enhances calibration performance, adversarial robustness, and out-of-distribution detection ability.
COMPUTER VISION AND IMAGE UNDERSTANDING
(2023)
Article
Geochemistry & Geophysics
Yunsheng Xiong, Peng Zhang, Yong Dou, Kele Xu, Xin Niu
Summary: Researchers have proposed a new deep learning framework that distills knowledge from an expert model to enable a student model to handle both hard categories and hard examples simultaneously. Experimental results demonstrate that this method outperforms existing baseline methods on three benchmark datasets.
IEEE GEOSCIENCE AND REMOTE SENSING LETTERS
(2022)
Article
Computer Science, Artificial Intelligence
Chang-Bin Zhang, Peng-Tao Jiang, Qibin Hou, Yunchao Wei, Qi Han, Zhen Li, Ming-Ming Cheng
Summary: The proposed Online Label Smoothing (OLS) strategy aims to improve classification performance of deep neural networks by generating more reliable soft labels, thereby enhancing the robustness of the model to noisy labels.
IEEE TRANSACTIONS ON IMAGE PROCESSING
(2021)
Article
Environmental Sciences
Gang Xu, Min Deng, Geng Sun, Ya Guo, Jie Chen
Summary: This paper proposes a knowledge distillation-based building extraction method to reduce the impact of noise on the model and improve the performance. The method utilizes the generalizable knowledge from large-scale noisy samples and accurate supervision from small-scale clean samples to train a teacher and student network. Experimental results show that the student network can alleviate the influence of noise labels and achieve accurate building extraction.
Article
Computer Science, Artificial Intelligence
Jiabin Liu, Biao Li, Minglong Lei, Yong Shi
Summary: This paper introduces a new learning paradigm called learning from complementary labels and proposes a novel learning framework that combines self-supervised learning and self-distillation. The framework achieves promising results by leveraging data and model information for complementary label learning.
Article
Computer Science, Artificial Intelligence
Runqing Jiang, Yan Yan, Jing-Hao Xue, Si Chen, Nannan Wang, Hanzi Wang
Summary: This article explores the issue of knowledge distillation (KD) with noisy labels and proposes a novel method called ambiguity-guided mutual label refinery KD (AML-KD) to train the student model in the presence of noisy labels. The AML-KD method introduces a label refinery framework and an ambiguity-aware weight estimation module to refine labels gradually and address the problem of ambiguous samples, achieving a high-accuracy and low-cost student model with label noise.
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
(2023)
Article
Computer Science, Artificial Intelligence
Yanyan Shi, Dianxi Shi, Ziteng Qiao, Zhen Wang, Yi Zhang, Shaowu Yang, Chunping Qiu
Summary: This paper proposes a method based on multi-granularity knowledge distillation and prototype consistency regularization (MDPCR) that achieves good performance even when previous training data is unavailable. The method captures multi-granularity by distilling features from three aspects and preserves the prototype of each old class. Extensive experiments confirm that MDPCR performs significantly better over exemplar-free methods and outperforms typical exemplar-based approaches.
Article
Computer Science, Artificial Intelligence
Xiaotong Yu, Shiding Sun, Yingjie Tian
Summary: The article introduces the difficulties and problems of partial label learning (PLL), and proposes an innovative multi-task framework to solve these problems. The framework combines self-supervision and self-distillation, and uses cross-sample knowledge and auxiliary self-supervised tasks to improve feature learning. Empirical results show that this method is effective in dealing with partially labeled data.
PATTERN RECOGNITION
(2024)
Article
Automation & Control Systems
Yiteng Pan, Fazhi He, Xiaohu Yan, Haoran Li
Summary: Motivated by the success of deep learning techniques, this paper introduces two typical methods of deep learning models developed for recommender systems: user-oriented autoencoder and item-oriented autoencoder. Studies show that the IAE model performs better in rating prediction tasks, while the UAE model performs better in top-N recommendation tasks. The authors propose a new SHAE method that combines the features learned by IAE and UAE models, achieving efficient recommendations.
ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE
(2021)
Article
Computer Science, Artificial Intelligence
Zhiqiang Bao, Zhenhua Huang, Jianping Gou, Lan Du, Kang Liu, Jingtao Zhou, Yunwen Chen
Summary: This paper proposes a novel framework called Teacher-Student Complementary Sample Contrastive Distillation (TSCSCD), which improves the performance of compact student models by addressing the challenges of weak supervision and overconfident predictions. TSCSCD outperforms recent state-of-the-art knowledge distillation techniques.
Article
Computer Science, Information Systems
Peng Liang, Weiwei Zhang, Junhuang Wang, Yufeng Guo
Summary: This paper proposes a concise and efficient Self-KD method called Neighbor Self-Knowledge Distillation (NSKD), which introduces teacher assistants into the Self-KD by adding auxiliary classifiers to the shallow part of the network. Experimental results demonstrate that NSKD outperforms other state-of-the-art Self-KD methods on multiple network models and datasets.
INFORMATION SCIENCES
(2024)
Article
Engineering, Electrical & Electronic
Naveen Paluru, Hariharan Ravishankar, Sharat Hegde, Phaneendra K. Yalavarthy
Summary: Optical coherence tomography (OCT) imaging has become a popular imaging modality for diagnosing retinal diseases. However, the varying speckle noise in OCT images hinders the performance of existing deep learning models. This study proposes a self distillation framework based on lightweight deep learning models for building more generalizable deep models for retinal disease diagnosis, which outperforms existing methods.
IEEE JOURNAL OF SELECTED TOPICS IN QUANTUM ELECTRONICS
(2023)
Article
Computer Science, Artificial Intelligence
Rui Lv, Dingheng Wang, Jiangbin Zheng, Zhao-Xu Yang
Summary: In this paper, the authors investigate tensor decomposition for neural network compression. They analyze the convergence and precision of tensor mapping theory, validate the rationality of tensor mapping and its superiority over traditional tensor approximation based on the Lottery Ticket Hypothesis. They propose an efficient method called 3D-KCPNet to compress 3D convolutional neural networks using the Kronecker canonical polyadic (KCP) tensor decomposition. Experimental results show that 3D-KCPNet achieves higher accuracy compared to the original baseline model and the corresponding tensor approximation model.
Article
Computer Science, Artificial Intelligence
Xiangkun He, Zhongxu Hu, Haohan Yang, Chen Lv
Summary: In this paper, a novel constrained multi-objective reinforcement learning algorithm is proposed for personalized end-to-end robotic control with continuous actions. The approach trains a single model using constraint design and a comprehensive index to achieve optimal policies based on user-specified preferences.
Article
Computer Science, Artificial Intelligence
Zhijian Zhuo, Bilian Chen, Shenbao Yu, Langcai Cao
Summary: In this paper, a novel method called Expansion with Contraction Method for Overlapping Community Detection (ECOCD) is proposed, which utilizes non-negative matrix factorization to obtain disjoint communities and applies expansion and contraction processes to adjust the degree of overlap. ECOCD is applicable to various networks with different properties and achieves high-quality overlapping community detection.
Article
Computer Science, Artificial Intelligence
Yizhe Zhu, Chunhui Zhang, Jialin Gao, Xin Sun, Zihan Rui, Xi Zhou
Summary: In this work, the authors propose a Contrastive Spatio-Temporal Distilling (CSTD) approach to improve the detection of high-compressed deepfake videos. The approach leverages spatial-frequency cues and temporal-contrastive alignment to fully exploit spatiotemporal inconsistency information.
Review
Computer Science, Artificial Intelligence
Laijin Meng, Xinghao Jiang, Tanfeng Sun
Summary: This paper provides a review of coverless steganographic algorithms, including the development process, known contributions, and general issues in image and video algorithms. It also discusses the security of coverless steganography from theoretical analysis to actual investigation for the first time.
Article
Computer Science, Artificial Intelligence
Yajie Bao, Tianwei Xing, Xun Chen
Summary: Visual question answering requires processing multi-modal information and effective reasoning. Neural-symbolic learning is a promising method, but current approaches lack uncertainty handling and can only provide a single answer. To address this, we propose a confidence based neural-symbolic approach that evaluates NN inferences and conducts reasoning based on confidence.
Article
Computer Science, Artificial Intelligence
Anh H. Vo, Bao T. Nguyen
Summary: Interior style classification is an interesting problem with potential applications in both commercial and academic domains. This project proposes a method named ISC-DeIT, which combines data-efficient image transformer architectures and knowledge distillation, to address the interior style classification problem. Experimental results demonstrate a significant improvement in predictive accuracy compared to other state-of-the-art methods.
Article
Computer Science, Artificial Intelligence
Shashank Kotyan, Danilo Vasconcellos Vargas
Summary: This article introduces a novel augmentation technique called Dynamic Scanning Augmentation to improve the accuracy and robustness of Vision Transformer (ViT). The technique leverages dynamic input sequences to adaptively focus on different patches, resulting in significant changes in ViT's attention mechanism. Experimental results demonstrate that Dynamic Scanning Augmentation outperforms ViT in terms of both robustness to adversarial attacks and accuracy against natural images.
Article
Computer Science, Artificial Intelligence
Hiba Alqasir, Damien Muselet, Christophe Ducottet
Summary: The article proposes a solution to improve the learning process of a classification network by providing shape priors, reducing the need for annotated data. The solution is tested on cross-domain digit classification tasks and a video surveillance application.
Article
Computer Science, Artificial Intelligence
Dexiu Ma, Mei Liu, Mingsheng Shang
Summary: This paper proposes a method using neural dynamics solvers to solve infinity-norm optimization problems. Two improved solvers are constructed and their effectiveness and superiority are demonstrated through theoretical analysis and simulation experiments.
Article
Computer Science, Artificial Intelligence
Francesco Gregoretti, Giovanni Pezzulo, Domenico Maisto
Summary: Active Inference is a computational framework that uses probabilistic inference and variational free energy minimization to describe perception, planning, and action. cpp-AIF is a header-only C++ library that provides a powerful tool for implementing Active Inference for Partially Observable Markov Decision Processes through multi-core computing. It is cross-platform and improves performance, memory management, and usability compared to existing software.
Article
Computer Science, Artificial Intelligence
Zelin Ying, Dawei Cheng, Cen Chen, Xiang Li, Peng Zhu, Yifeng Luo, Yuqi Liang
Summary: This paper proposes a novel stock market trends prediction framework called SMART, which includes a self-supervised stock technical data sequence embedding model S3E. By training with multiple self-supervised auxiliary tasks, the model encodes stock technical data sequences into embeddings and uses the learned sequence embeddings for predicting stock market trends. Extensive experiments on China A-Shares market and NASDAQ market prove the high effectiveness of our model in stock market trends prediction, and its effectiveness is further validated in real-world applications in a leading financial service provider in China.
Article
Computer Science, Artificial Intelligence
Hao Li, Hao Jiang, Dongsheng Ye, Qiang Wang, Liang Du, Yuanyuan Zeng, Liu Yuan, Yingxue Wang, C. Chen
Summary: DHGAT1, a dynamic hyperbolic graph attention network, utilizes hyperbolic metric properties to embed dynamic graphs. It employs a spatiotemporal self-attention mechanism and weighted node representations, resulting in excellent performance in link prediction tasks.
Article
Computer Science, Artificial Intelligence
Jiehui Huang, Zhenchao Tang, Xuedong He, Jun Zhou, Defeng Zhou, Calvin Yu-Chian Chen
Summary: This study proposes a progressive learning multi-scale feature blending model for image deraining tasks. The model utilizes detail dilation and texture extraction to improve the restoration of rainy images. Experimental results show that the model achieves near state-of-the-art performance in rain removal tasks and exhibits better rain removal realism.
Article
Computer Science, Artificial Intelligence
Lizhi Liu, Zilin Gao, Yinhe Wang, Yongfu Li
Summary: This paper proposes a novel discrete-time interconnected model for depicting complex dynamical networks. The model consists of nodes and edges subsystems, which consider the dynamic characteristic of both nodes and edges. By designing control strategies and coupling modes, the stabilization and synchronization of the network are achieved. Simulation results demonstrate the effectiveness of the proposed methods.