Article
Environmental Sciences
Abhirup Dikshit, Biswajeet Pradhan
Summary: Accurately predicting natural hazards, especially drought, is challenging. Including climatic variables in data-driven prediction models improves accuracy. Using explainable artificial intelligence models can help understand local interactions during different drought conditions and periods.
SCIENCE OF THE TOTAL ENVIRONMENT
(2021)
Article
Computer Science, Artificial Intelligence
Liangchen Xu, Chonghui Guo
Summary: Survival analysis is widely used in various fields to model the relationship between the time of an event and related features, but traditional models lack nonlinearity and interpretability. To address these limitations, we propose an interpretable deep survival analysis model called CoxNAM, based on the Cox proportional hazards model and neural additive model. The model provides survival functions, shape functions of features, and feature importance while predicting the probability of the event occurrence. Experimental results demonstrate its performance and interpretability, outperforming traditional and machine learning-based models. The proposed method shows promising practical applications in survival analysis and decision-making.
EXPERT SYSTEMS WITH APPLICATIONS
(2023)
Article
Computer Science, Artificial Intelligence
Mara Graziani, Lidia Dutkiewicz, Davide Calvaresi, Jose Pereira Amorim, Katerina Yordanova, Mor Vered, Rahul Nair, Pedro Henriques Abreu, Tobias Blanke, Valeria Pulignano, John O. Prior, Lode Lauwaert, Wessel Reijers, Adrien Depeursinge, Vincent Andrearczyk, Henning Muller
Summary: Since its emergence in the 1960s, Artificial Intelligence (AI) has been widely applied to various technology products and fields. Machine learning, as a major part of current AI solutions, achieves high performance on various tasks through learning from data and experience. However, the interpretability of AI models, especially deep neural networks, is often challenging. Different domains have different requirements for interpretability and tools for debugging and validating models. In this paper, the authors propose a unified terminology and definition of interpretability in AI systems, aiming to improve clarity and efficiency in the regulation of ethical and reliable AI development, and to facilitate communication across interdisciplinary areas of AI.
ARTIFICIAL INTELLIGENCE REVIEW
(2023)
Article
Computer Science, Artificial Intelligence
Mateusz Krzyzinski, Mikolaj Spytek, Hubert Baniecki, Przemyslaw Biecek
Summary: Machine and deep learning survival models have comparable or better prediction capabilities for time-to-event compared to classical statistical learning methods, but they are too complex for human interpretation. This paper introduces SurvSHAP(t), the first time-dependent explanation method for interpreting survival black-box models, based on SHapley Additive exPlanations with solid theoretical foundations. The proposed method aims to improve precision diagnostics and support decision-making by domain experts.
KNOWLEDGE-BASED SYSTEMS
(2023)
Article
Computer Science, Artificial Intelligence
Krasymyr Tretiak, Scott Ferson
Summary: Data quality is crucial in engineering applications and projects, and data collection procedures may not always utilize the most precise instruments and protocols. The question of whether to pool data of differing qualities or exclude imprecise data is examined, with concerns about depreciating overall quality or increasing uncertainty. Simulation results demonstrate when it is advisable to combine precise and imprecise data using different mathematical representations of imprecision.
INTERNATIONAL JOURNAL OF APPROXIMATE REASONING
(2023)
Article
Computer Science, Information Systems
Usama Masood, Hasan Farooq, Ali Imran, Adnan Abu-Dayya
Summary: In modern wireless communication systems, radio propagation modeling for pathloss estimation is crucial for system design and optimization. Existing empirical models have limitations in capturing the characteristics of different propagation environments. To address this issue, we propose a Machine Learning (ML)-based model that utilizes novel predictors to estimate pathloss. Our evaluation shows that the Light Gradient Boosting Machine (LightGBM) algorithm outperforms other ML algorithms, providing a 65% increase in prediction accuracy compared to empirical models and a 13x decrease in prediction time compared to ray-tracing. We also use the SHapley Additive exPlanations (SHAP) method to gain insights for network configuration tuning, data enrichment, and building a lighter ML-based propagation model for low-latency use-cases.
IEEE TRANSACTIONS ON MOBILE COMPUTING
(2023)
Article
Computer Science, Artificial Intelligence
Michela Proietti, Alessio Ragno, Biagio La Rosa, Rino Ragno, Roberto Capobianco
Summary: In this work, concept whitening is applied to graph neural networks to improve both classification performance and interpretability. By identifying key concepts and structural parts of molecules, explanations are provided for the predictions.
Article
Computer Science, Information Systems
Lev Utkin, Vladimir S. Zaborovsky, Maxim S. Kovalev, Andrei Konstantinov, Natalia A. Politaeva, Alexey A. Lukashin
Summary: The UncSurvEx method is proposed to interpret the uncertainty of predictions provided by machine learning survival models by approximating the uncertainty measure of a local black-box survival model prediction with the Cox proportional hazards model. The method involves computing the distance between survival functions and solving an unconstrained non-convex optimization problem, leading to a new way of interpreting prediction uncertainty.
Article
Engineering, Electrical & Electronic
Theerasarn Pianpanit, Sermkiat Lolak, Phattarapong Sawangjai, Thapanun Sudhawiyangkul, Theerawit Wilaiprasitporn
Summary: This study explores the recognition of Parkinson's disease using SPECT images and deep learning methods, introducing the procedure and results of selecting interpretation methods suitable for PD recognition models. The evaluation reveals that guided backpropagation and SHAP interpretation methods exhibit good performance in PD recognition.
IEEE SENSORS JOURNAL
(2021)
Article
Computer Science, Artificial Intelligence
Miroslav Hudec, Erika Minarikova, Radko Mesiar, Anna Saranti, Andreas Holzinger
Summary: The study introduced a novel classification method that empowers domain experts to choose important observations for attributes and utilizes function variability for machine learning opportunities. Demonstrated the research steps of human-in-the-loop interactive machine learning with aggregation functions.
KNOWLEDGE-BASED SYSTEMS
(2021)
Article
Computer Science, Artificial Intelligence
Lev Utkin, Egor D. Satyukov, Andrei Konstantinov
Summary: This paper proposes a Neural Additive Model (NAM) extension called SurvNAM, which is used to explain predictions of a black-box machine learning survival model. The proposed method trains the network using a specific loss function and approximates the black-box model using an extended Cox proportional hazards model. SurvNAM allows for both local and global explanations, and its efficiency is demonstrated through numerous numerical experiments.
Article
Mathematical & Computational Biology
Weichi Yao, Halina Frydman, Jeffrey S. Simonoff
Summary: Interval-censored data analysis is important for time-to-event response in biomedical statistics. The proposed survival forest method performs effectively in various scenarios, influenced by tuning parameters. Monte Carlo simulations show its performance compared to other methods in different model structures.
Article
Mathematical & Computational Biology
Sarah F. McGough, Devin Incerti, Svetlana Lyalina, Ryan Copping, Balasubramanian Narasimhan, Robert Tibshirani
Summary: The prevalence of high-dimensional data in the medical field has led to challenges in statistical modeling, particularly when dealing with left-truncated survival data. Feature selection or penalized methods are often used to address overfitting issues, highlighting the importance of adjusting for left truncation in survival modeling.
STATISTICS IN MEDICINE
(2021)
Review
Genetics & Heredity
Martin Treppner, Harald Binder, Moritz Hess
Summary: Deep generative models have been applied to learn the underlying structure of omics data, and this paper provides an introduction and overview of such techniques, with a focus on their use in single-cell gene expression data. By utilizing methods like variational auto-encoders, deep generative models can offer low dimensional latent representations that aid in understanding the relationships between observed gene expressions and experimental factors or phenotypes. Additionally, these models can generate synthetic observations to assess the uncertainty in the learned representations. However, the interpretability of deep generative models can be challenging due to their neural network building blocks. The paper also presents approaches that can be used to infer the relationship between learned latent variables and observed variables, making deep learning approaches more interpretable. The utility of the discussed methods is demonstrated through an application with single-cell gene expression data.
Article
Mathematics, Applied
Malkhaz Shashiashvili
Summary: This paper establishes a simple explicit upper bound for the quadratic risk of the Grenander estimator, and proves this conclusion is a direct result of an inequality valid with probability one.
PROCEEDINGS OF THE ROYAL SOCIETY OF EDINBURGH SECTION A-MATHEMATICS
(2021)
Article
Computer Science, Artificial Intelligence
Andrei Konstantinov, Lev Utkin
Summary: A new multi-attention based method is proposed for solving the MIL problem, which efficiently processes different types of patches and provides diverse feature representation. It takes into account the neighboring patches or instances to provide accurate classification results.
NEURAL COMPUTING & APPLICATIONS
(2022)
Article
Computer Science, Artificial Intelligence
Lev Utkin, Egor D. Satyukov, Andrei Konstantinov
Summary: This paper proposes a Neural Additive Model (NAM) extension called SurvNAM, which is used to explain predictions of a black-box machine learning survival model. The proposed method trains the network using a specific loss function and approximates the black-box model using an extended Cox proportional hazards model. SurvNAM allows for both local and global explanations, and its efficiency is demonstrated through numerous numerical experiments.
Article
Computer Science, Artificial Intelligence
Andrei Konstantinov, Lev Utkin
Summary: A system called AFEX is proposed for explaining the predictions of machine learning black-box models on tabular data. The main advantage of AFEX is its ability to identify pairwise interactions between features without the need for retraining neural networks.
INTERNATIONAL JOURNAL OF DATA SCIENCE AND ANALYTICS
(2023)
Article
Computer Science, Artificial Intelligence
Lev Utkin, Andrei Konstantinov
Summary: This paper proposes a new approach called Attention-Based Random Forest (ABRF) and its modifications for regression and classification tasks. The main idea is to assign attention weights with trainable parameters to decision trees in a specific way. The approach is based on the representation of Nadaraya-Watson kernel regression as a random forest. Three modifications are proposed, including the use of Huber's contamination model and gradient-based algorithms. Experimental results on various datasets demonstrate the effectiveness of the proposed method.
Article
Computer Science, Artificial Intelligence
Lev Utkin, Andrey Ageev, Andrei Konstantinov, Vladimir Muliukha
Summary: A new modification of the isolation forest called attention-based isolation forest (ABIForest) is proposed, which incorporates an attention mechanism to improve anomaly detection. Attention weights are assigned to each path of trees with learnable parameters, and Huber's contamination model is used to define the attention weights. ABIForest is the first modification of the isolation forest to incorporate an attention mechanism in a simple way, and it outperforms other methods in numerical experiments.
Article
Computer Science, Artificial Intelligence
Lev Utkin, Andrei Konstantinov
Summary: This paper proposes ensemble-based modifications to simplify the SHapley Additive exPlanations (SHAP) method for the local explanation of a black-box model. The modifications approximate the SHAP by ensembles with a smaller number of features. Three modifications are proposed, namely ER-SHAP, ERW-SHAP, and ER-SHAP-RF. Numerical experiments demonstrate the effectiveness and local interpretability of these modifications.
Article
Computer Science, Artificial Intelligence
Lev V. Utkin, Andrei V. Konstantinov, Stanislav R. Kirpichenko
Summary: New models of random forests using both attention and self-attention mechanisms are proposed. These models are extensions of attention-based random forests, which combine Nadaraya-Watson kernel regression and Huber's contamination model. The self-attention aims to capture dependencies between tree predictions and remove noise or anomalous predictions. The attention weights are trained using quadratic or linear optimization, and modifications to the self-attention are proposed and compared. The models are verified and compared with other random forest models using multiple datasets, with publicly available code.
PROGRESS IN ARTIFICIAL INTELLIGENCE
(2023)
Article
Computer Science, Artificial Intelligence
Andrei Konstantinov, Stanislav Kirpichenko, Lev Utkin
Summary: This paper proposes a new method called TNW-CATE (the Trainable Nadaraya-Watson regression for CATE) for estimating the conditional average treatment effect. TNW-CATE utilizes the Nadaraya-Watson regression and a weight sharing neural network to train the kernels, allowing it to predict the outcomes of patients from control and treatment groups. The proposed approach is similar to transfer learning when the domains of source and target data are similar, but the tasks are different. Multiple numerical simulation experiments are conducted to illustrate TNW-CATE and compare it with other well-known methods.
Article
Computer Science, Interdisciplinary Applications
Andrei Konstantinov, Lev Utkin, Vladimir Muliukha
Summary: This paper introduces LARF, a new model of attention-based random forests. The model incorporates a two-level attention mechanism, with leaf attention applied to each leaf and tree attention dependent on leaf attention. The softmax operation in attention is replaced by a weighted sum of softmax operations with different parameters. Attention parameters are trained by solving a quadratic optimization problem. The proposed algorithms are evaluated through numerical experiments with real datasets and the code is available.
Article
Computer Science, Artificial Intelligence
Andrei V. Konstantinov, Lev V. Utkin
Summary: A new ensemble-based model, which adopts uniformly generated axis-parallel hyper-rectangles as base models, is proposed in this paper. By considering training examples inside and outside each rectangle, the HRBMs are incorporated into the gradient boosting machine to construct effective ensemble-based models and avoid overfitting. Additionally, a new regularization method called step height penalty is studied, and a simple interpretation approach based on SHAP method is proposed.
NEURAL COMPUTING & APPLICATIONS
(2023)
Article
Computer Science, Artificial Intelligence
Andrei Konstantinov, Lev Utkin, Vladimir Muliukha
Summary: This paper proposes a new model called Soft Tree Ensemble Multiple Instance Learning, based on random forest, to solve the Multiple Instance Learning problem under small tabular data. It introduces a new type of soft decision trees, which are similar to soft oblique trees but have fewer trainable parameters. The trees are transformed into neural networks that approximate the tree functions, and the instance and bag embeddings are aggregated using the attention mechanism. The proposed model, including the soft decision trees, neural networks, attention mechanism, and classifier, is trained in an end-to-end manner. Numerical experiments demonstrate that the model outperforms existing multiple instance learning models, and the code implementation is publicly available.
Article
Computer Science, Information Systems
Maxim Kovalev, Lev Utkin, Frank Coolen, Andrei Konstantinov
Summary: A method for counterfactual explanation of machine learning survival models is proposed in this paper, introducing a condition to establish the difference between survival functions of the original example and the counterfactual. When the black-box model is the Cox model, the counterfactual explanation problem can be simplified to a standard convex optimization problem with linear constraints, while for other black-box models, the Particle Swarm Optimization algorithm is recommended. Numerical experiments with real and synthetic data validate the effectiveness of the proposed method.
Article
Computer Science, Information Systems
Lev Utkin, Vladimir S. Zaborovsky, Maxim S. Kovalev, Andrei Konstantinov, Natalia A. Politaeva, Alexey A. Lukashin
Summary: The UncSurvEx method is proposed to interpret the uncertainty of predictions provided by machine learning survival models by approximating the uncertainty measure of a local black-box survival model prediction with the Cox proportional hazards model. The method involves computing the distance between survival functions and solving an unconstrained non-convex optimization problem, leading to a new way of interpreting prediction uncertainty.
Article
Computer Science, Artificial Intelligence
Hamdan Abdellatef, Lina J. Karam
Summary: This paper proposes performing the learning and inference processes in the compressed domain to reduce computational complexity and improve speed of neural networks. Experimental results show that modified ResNet-50 in the compressed domain is 70% faster than traditional spatial-based ResNet-50 while maintaining similar accuracy. Additionally, a preprocessing step with partial encoding is suggested to improve resilience to distortions caused by low-quality encoded images. Training a network with highly compressed data can achieve good classification accuracy with significantly reduced storage requirements.
Article
Computer Science, Artificial Intelligence
Victor R. Barradas, Yasuharu Koike, Nicolas Schweighofer
Summary: Inverse models are essential for human motor learning as they map desired actions to motor commands. The shape of the error surface and the distribution of targets in a task play a crucial role in determining the speed of learning.
Article
Computer Science, Artificial Intelligence
Ting Zhou, Hanshu Yan, Jingfeng Zhang, Lei Liu, Bo Han
Summary: We propose a defense strategy that reduces the success rate of data poisoning attacks in downstream tasks by pre-training a robust foundation model.
Article
Computer Science, Artificial Intelligence
Hao Sun, Li Shen, Qihuang Zhong, Liang Ding, Shixiang Chen, Jingwei Sun, Jing Li, Guangzhong Sun, Dacheng Tao
Summary: In this paper, the convergence rate of AdaSAM in the stochastic non-convex setting is analyzed. Theoretical proof shows that AdaSAM has a linear speedup property and decouples the stochastic gradient steps with the adaptive learning rate and perturbed gradient. Experimental results demonstrate that AdaSAM outperforms other optimizers in terms of performance.
Article
Computer Science, Artificial Intelligence
Juntong Yun, Du Jiang, Li Huang, Bo Tao, Shangchun Liao, Ying Liu, Xin Liu, Gongfa Li, Disi Chen, Baojia Chen
Summary: In this study, a dual manipulator grasping detection model based on the Markov decision process is proposed. By parameterizing the grasping detection model of dual manipulators using a cross entropy convolutional neural network and a full convolutional neural network, stable grasping of complex multiple objects is achieved. Robot grasping experiments were conducted to verify the feasibility and superiority of this method.
Article
Computer Science, Artificial Intelligence
Miaohui Zhang, Kaifang Li, Jianxin Ma, Xile Wang
Summary: This paper proposes an unsupervised person re-identification (Re-ID) method that uses two asymmetric networks to generate pseudo-labels for each other by clustering and updates and optimizes the pseudo-labels through alternate training. It also designs similarity compensation and similarity suppression based on the camera ID of pedestrian images to optimize the similarity measure. Extensive experiments show that the proposed method achieves superior performance compared to state-of-the-art unsupervised person re-identification methods.
Article
Computer Science, Artificial Intelligence
Florian Bacho, Dominique Chu
Summary: This paper proposes a new approach called the Forward Direct Feedback Alignment algorithm for supervised learning in deep neural networks. By combining activity-perturbed forward gradients, direct feedback alignment, and momentum, this method achieves better performance and convergence speed compared to other local alternatives to backpropagation.
Article
Computer Science, Artificial Intelligence
Xiaojian Ding, Yi Li, Shilin Chen
Summary: This research paper addresses the limitations of recursive feature elimination (RFE) and its variants in high-dimensional feature selection tasks. The proposed algorithms, which introduce a novel feature ranking criterion and an optimal feature subset evaluation algorithm, outperform current state-of-the-art methods.
Article
Computer Science, Artificial Intelligence
Naoko Koide-Majima, Shinji Nishimoto, Kei Majima
Summary: Visual images observed by humans can be reconstructed from brain activity, and the visualization of arbitrary natural images from mental imagery has been achieved through an improved method. This study provides a unique tool for directly investigating the subjective contents of the brain.
Article
Computer Science, Artificial Intelligence
Huanjie Tao, Qianyue Duan
Summary: In this paper, a hierarchical attention network with progressive feature fusion is proposed for facial expression recognition (FER), addressing the challenges posed by pose variation, occlusions, and illumination variation. The model achieves enhanced performance by aggregating diverse features and progressively enhancing discriminative features.
Article
Computer Science, Artificial Intelligence
Zhenyi Wang, Pengfei Yang, Linwei Hu, Bowen Zhang, Chengmin Lin, Wenkai Lv, Quan Wang
Summary: In the face of the complex landscape of deep learning, we propose a novel subgraph-level performance prediction method called SLAPP, which combines graph and operator features through an innovative graph neural network called EAGAT, providing accurate performance predictions. In addition, we introduce a mixed loss design with dynamic weight adjustment to improve predictive accuracy.
Article
Computer Science, Artificial Intelligence
Yiyang Yin, Shuangling Luo, Jun Zhou, Liang Kang, Calvin Yu-Chian Chen
Summary: Medical image segmentation is crucial for modern healthcare systems, especially in reducing surgical risks and planning treatments. Transanal total mesorectal excision (TaTME) has become an important method for treating colon and rectum cancers. Real-time instance segmentation during TaTME surgeries can assist surgeons in minimizing risks. However, the dynamic variations in TaTME images pose challenges for accurate instance segmentation.
Article
Computer Science, Artificial Intelligence
Teng Cheng, Lei Sun, Junning Zhang, Jinling Wang, Zhanyang Wei
Summary: This study proposes a scheme that combines the start-stop point signal features for wideband multi-signal detection, called Fast Spectrum-Size Self-Training network (FSSNet). By utilizing start-stop points to build the signal model, this method successfully solves the difficulty of existing deep learning methods in detecting discontinuous signals and achieves satisfactory detection speed.
Article
Computer Science, Artificial Intelligence
Wenming Wu, Xiaoke Ma, Quan Wang, Maoguo Gong, Quanxue Gao
Summary: The layer-specific modules in multi-layer networks are critical for understanding the structure and function of the system. However, existing methods fail to accurately characterize and balance the connectivity and specificity of these modules. To address this issue, a joint learning graph clustering algorithm (DRDF) is proposed, which learns the deep representation and discriminative features of the multi-layer network, and balances the connectivity and specificity of the layer-specific modules through joint learning.
Article
Computer Science, Artificial Intelligence
Guanghui Yue, Guibin Zhuo, Weiqing Yan, Tianwei Zhou, Chang Tang, Peng Yang, Tianfu Wang
Summary: This paper proposes a novel boundary uncertainty aware network (BUNet) for precise and robust colorectal polyp segmentation. BUNet utilizes a pyramid vision transformer encoder to learn multi-scale features and incorporates a boundary exploration module (BEM) and a boundary uncertainty aware module (BUM) to handle boundary areas. Experimental results demonstrate that BUNet outperforms other methods in terms of performance and generalization ability.
