Article
Computer Science, Artificial Intelligence
Dexuan Xu, Huashi Zhu, Yu Huang, Zhi Jin, Weiping Ding, Hang Li, Menglong Ran
Summary: In this paper, a vision-knowledge fusion model based on medical images and knowledge graphs is proposed to fully utilize high-quality data from different diseases and languages. The model automatically constructs domain-specific knowledge graphs based on medical standards, fuses image and knowledge using a knowledge-based attention mechanism, and restores fine-grained knowledge through a triples restoration module. Experimental results show that the model outperforms previous benchmark methods and achieves excellent evaluation scores on two different diseases datasets. The interpretability and clinical usefulness of the model are validated, and it can be generalized to multiple domains and different diseases.
INFORMATION FUSION
(2023)
Article
Chemistry, Multidisciplinary
Dehai Zhang, Anquan Ren, Jiashu Liang, Qing Liu, Haoxing Wang, Yu Ma
Summary: This paper focuses on the automatic generation of medical reports from chest X-ray images. By constructing associations based on a knowledge graph and using a graph neural network, disease situational representations with prior knowledge are generated, and radiology reports are generated using self-supervised learning. Experimental results demonstrate that this method outperforms existing methods in performance.
APPLIED SCIENCES-BASEL
(2022)
Article
Computer Science, Artificial Intelligence
Shuxin Yang, Xian Wu, Shen Ge, Zhuozhao Zheng, S. Kevin Zhou, Li Xiao
Summary: This study presents an automatic, multi-modal approach for generating radiology reports from chest x-ray images. The approach consists of a learned knowledge base module and a multi-modal alignment module, which utilize the correlation between the descriptions in radiology reports and specific information in the x-ray images. The proposed model is evaluated using metrics from natural language generation and clinic efficacy, and it shows improved performance compared to state-of-the-art methods.
MEDICAL IMAGE ANALYSIS
(2023)
Article
Computer Science, Artificial Intelligence
Guoquan Dai, Xizhao Wang, Xiaoying Zou, Chao Liu, Si Cen
Summary: This article introduces a new multi-relational graph attention network (MRGAT) that can calculate the importance of different neighboring nodes in a knowledge graph, effectively improving the performance of the network.
Article
Computer Science, Artificial Intelligence
Ivona Najdenkoska, Xiantong Zhen, Marcel Worring, Ling Shao
Summary: Automating report generation for medical imaging using probabilistic variational topic inference can generate reports with novel sentence structure, rather than mere copies of training samples, while achieving comparable performance to state-of-the-art methods.
MEDICAL IMAGE ANALYSIS
(2022)
Article
Computer Science, Artificial Intelligence
Xu Yu, Qinglong Peng, Feng Jiang, Junwei Du, Hongtao Liang, Jinhuan Liu
Summary: Recently, cross-domain collaborative filtering (CDCF) has gained popularity in addressing the data sparsity issue in recommendation systems. This paper proposes a Multi-head Attention and Knowledge Graph Based Dual Target Graph Collaborative Filtering Network (MAKG-DTGCF) to improve the recommendation performance of both target and source domains. The MAKG-DTGCF model utilizes multi-head attention for adaptive transfer and fusion of user features in multiple representation subspaces, and enhances item representation through alignment with knowledge graphs. Experimental results demonstrate that the MAKG-DTGCF model outperforms state-of-the-art models in HR and NDCG metrics.
NEURAL PROCESSING LETTERS
(2023)
Article
Computer Science, Artificial Intelligence
Huajuan Duan, Peiyu Liu, Qi Ding
Summary: Knowledge graphs (KGs) provide rich external structures and semantic information for recommendation systems. Existing methods process triplets independently, failing to capture the complex and implicit relation information. To address this, we propose a relation-fused multi-head attention network (RFAN) that integrates relations into an attention network in a KG, effectively capturing user preferences. Experimental results demonstrate that RFAN outperforms other methods.
APPLIED INTELLIGENCE
(2023)
Article
Engineering, Biomedical
Xiulong Yi, You Fu, Rong Hua, Ruiqing Liu, Hao Zhang
Summary: Recently, the automatic radiology report generation system has gained attention. Compared to image captioning, radiology report generation faces greater challenges due to data biases. Our proposed unsupervised disease tags model achieves state-of-the-art performance and provides more accurate abnormal findings.
BIOMEDICAL SIGNAL PROCESSING AND CONTROL
(2023)
Article
Biology
Sheng Zhang, Chuan Zhou, Leiting Chen, Zhiheng Li, Yuan Gao, Yongqi Chen
Summary: Automated radiology report generation is a popular method for alleviating the workload of radiologists and reducing misdiagnosis. However, existing approaches lack visual prior and alignment between images and texts. To address these issues, this study proposes a Visual Prior-based Cross-modal Alignment Network, which uses contrastive attention to extract visual prior and a cross-modal alignment network to align images and texts. Experimental results on benchmark datasets demonstrate that the proposed model outperforms state-of-the-art models in terms of various metrics.
COMPUTERS IN BIOLOGY AND MEDICINE
(2023)
Article
Computer Science, Interdisciplinary Applications
Gaurav O. Gajbhiye, Abhijeet V. Nandedkar, Ibrahima Faye
Summary: This research work was supported by the joint collaboration between the Computer Vision and Pattern Vision Lab at Shri Guru Gobind Singhji Institute of Engineering and Technology, Nanded, India, and the Center for Intelligent Signal and Imaging Research at Universiti Teknologi PETRONAS, Seri Iskandar, Malaysia, under the International Grant 015ME0-018.
COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE
(2022)
Article
Computer Science, Artificial Intelligence
Ting Ma, Shangwen Lv, Longtao Huang, Songlin Hu
Summary: This paper presents a novel model HiAM for knowledge graph multi-hop reasoning, which utilizes predecessor paths and different granularities of information to conduct deep reasoning and achieves competitive performance.
Article
Computer Science, Artificial Intelligence
Zhifei Li, Yue Zhao, Yan Zhang, Zhaoli Zhang
Summary: This paper proposes a novel heterogeneous graph neural network framework based on a hierarchical attention mechanism for modeling knowledge graphs. The proposed model achieves outstanding performance on various heterogeneous graph tasks.
KNOWLEDGE-BASED SYSTEMS
(2022)
Article
Engineering, Electrical & Electronic
Caozhi Shang, Shaoguo Cui, Tiansong Li, Xi Wang, Yongmei Li, Jingfeng Jiang
Summary: Medical imaging plays an important role in clinical workflows, but automatically generating radiology reports faces challenges such as data biases and decoder limitations. To address these issues, we propose Multi-modal Adaptive Transformer (MATNet), a system that combines natural language processing and machine learning techniques to create fluent and accurate radiology reports.
IEEE SIGNAL PROCESSING LETTERS
(2022)
Article
Biochemistry & Molecular Biology
Liyi Yu, Zhaochun Xu, Meiling Cheng, Weizhong Lin, Wangren Qiu, Xuan Xiao
Summary: In this study, a deep learning framework called MSEDDI is proposed to predict drug-drug interaction events by comprehensively considering multi-scale embedding representations of the drug. The experimental results demonstrate that MSEDDI outperforms other existing methods in terms of prediction performance and exhibits stable performance in a broader sample set.
INTERNATIONAL JOURNAL OF MOLECULAR SCIENCES
(2023)
Article
Computer Science, Artificial Intelligence
Jiadong Yan, Yuzhong Chen, Zhenxiang Xiao, Shu Zhang, Mingxin Jiang, Tianqi Wang, Tuo Zhang, Jinglei Lv, Benjamin Becker, Rong Zhang, Dajiang Zhu, Junwei Han, Dezhong Yao, Keith M. Kendrick, Tianming Liu, Xi Jiang
Summary: In this study, a novel Multi-Head Guided Attention Graph Neural Network (Multi-Head GAGNN) was proposed to model both spatial and temporal patterns of holistic functional brain networks. The results showed that the Multi-Head GAGNN outperformed other state-of-the-art models in modeling brain function and predicting cognitive behavioral measures in individuals.
MEDICAL IMAGE ANALYSIS
(2022)
Article
Computer Science, Interdisciplinary Applications
Li Xiao, Chunlong Luo, Tianqi Yu, Yufan Luo, Manqing Wang, Fuhai Yu, Yinhao Li, Chan Tian, Jie Qiao
IEEE TRANSACTIONS ON MEDICAL IMAGING
(2020)
Article
Computer Science, Interdisciplinary Applications
Qingsong Yao, Li Xiao, Peihang Liu, S. Kevin Zhou
Summary: The study introduces a label-free approach for segmenting COVID-19 lesions in CT via voxel-level anomaly modeling, reducing the burden of data annotation. By learning patterns of normal tissues, a network capable of distinguishing normal tissues from COVID-19 lesions was established.
IEEE TRANSACTIONS ON MEDICAL IMAGING
(2021)
Article
Cell Biology
Liqing Liu, Shuxin Yang, Yang Liu, Xixia Li, Junjie Hu, Li Xiao, Tao Xu
Summary: In this study, a tool called DeepContact was developed for optimizing organelle segmentation and contact analysis using label-free electron microscopy (EM) through deep learning algorithms. DeepContact showed high efficiency and flexibility, capable of accommodating various organelle morphologies and identifying contacts of different widths. The study revealed previously unidentified coordinated rearrangements of organelles and a subtle wave of interaction between the endoplasmic reticulum (ER) and mitochondria in Sertoli cells during the seminiferous epithelial cycle, demonstrating the potential of DeepContact in bridging MCS dynamics to physiological and pathological processes.
JOURNAL OF CELL BIOLOGY
(2022)
Article
Computer Science, Artificial Intelligence
Shuxin Yang, Xian Wu, Shen Ge, Zhuozhao Zheng, S. Kevin Zhou, Li Xiao
Summary: This study presents an automatic, multi-modal approach for generating radiology reports from chest x-ray images. The approach consists of a learned knowledge base module and a multi-modal alignment module, which utilize the correlation between the descriptions in radiology reports and specific information in the x-ray images. The proposed model is evaluated using metrics from natural language generation and clinic efficacy, and it shows improved performance compared to state-of-the-art methods.
MEDICAL IMAGE ANALYSIS
(2023)
Article
Computer Science, Interdisciplinary Applications
Yijie Peng, Li Xiao, Bernd Heidergott, L. Jeff Hong, Henry Lam
Summary: A new method for computing the gradients of artificial neural networks was proposed, which bypasses the continuity requirement of traditional methods by injecting artificial noises into signals. The method shows similar computational complexity and more transparent formulas compared to traditional methods. Additionally, a likelihood ratio-based method was developed to train more general ANNs and improve their robustness.
INFORMS JOURNAL ON COMPUTING
(2022)
Article
Computer Science, Artificial Intelligence
Hong Liu, Dong Wei, Donghuan Lu, Xiaoying Tang, Liansheng Wang, Yefeng Zheng
Summary: This study proposes a framework based on hybrid 2D-3D convolutional neural networks for obtaining continuous 3D retinal layer surfaces from OCT volumes. The framework works well with both full and sparse annotations and utilizes alignment displacement vectors and layer segmentation to align the B-scans and segment the layers. Experimental results show that the framework outperforms state-of-the-art 2D deep learning methods in terms of layer segmentation accuracy and cross-B-scan 3D continuity.
MEDICAL IMAGE ANALYSIS
(2024)
Article
Computer Science, Artificial Intelligence
Simon Oxenford, Ana Sofia Rios, Barbara Hollunder, Clemens Neudorfer, Alexandre Boutet, Gavin J. B. Elias, Jurgen Germann, Aaron Loh, Wissam Deeb, Bryan Salvato, Leonardo Almeida, Kelly D. Foote, Robert Amaral, Paul B. Rosenberg, David F. Tang-Wai, David A. Wolk, Anna D. Burke, Marwan N. Sabbagh, Stephen Salloway, M. Mallar Chakravarty, Gwenn S. Smith, Constantine G. Lyketsos, Michael S. Okun, William S., Zoltan Mari, Francisco A. Ponce, Andres Lozano, Wolf-Julian Neumann, Bassam Al-Fatly, Andreas Horn
Summary: Spatial normalization is a method to map subject brain images to an average template brain, allowing comparison of brain imaging results. We introduce a novel tool called WarpDrive, which enables manual refinements of image alignment after automated registration. The tool improves accuracy of data representation and aids in understanding patient outcomes.
MEDICAL IMAGE ANALYSIS
(2024)
Article
Computer Science, Artificial Intelligence
Ricards Marcinkevics, Patricia Reis Wolfertstetter, Ugne Klimiene, Kieran Chin-Cheong, Alyssia Paschke, Julia Zerres, Markus Denzinger, David Niederberger, Sven Wellmann, Ece Ozkan, Christian Knorr, Julia E. Vogt
Summary: This study presents interpretable machine learning models for predicting the diagnosis, management, and severity of suspected appendicitis using ultrasound images. The proposed models utilize concept bottleneck models (CBM) that facilitate interpretation and intervention by clinicians, without compromising performance or requiring time-consuming image annotation.
MEDICAL IMAGE ANALYSIS
(2024)
Article
Computer Science, Artificial Intelligence
Jian-Qing Zheng, Ziyang Wang, Baoru Huang, Ngee Han Lim, Bartlomiej W. Papiez
Summary: This article introduces a new method for medical image registration, which utilizes a separable motion backbone and a residual aligner module to better handle the discontinuous motion of multiple neighboring objects. The proposed method achieves excellent registration results on abdominal CT scans and lung CT scans.
MEDICAL IMAGE ANALYSIS
(2024)
Article
Computer Science, Artificial Intelligence
Xiangqiong Wu, Guanghua Tan, Hongxia Luo, Zhilun Chen, Bin Pu, Shengli Li, Kenli Li
Summary: This study develops a user-friendly framework for the automated diagnosis of thyroid nodules in ultrasound videos, simulating the diagnostic workflow of radiologists. By interpreting image characteristics and modeling temporal contextual information, the efficiency and generalizability of the diagnosis can be improved.
MEDICAL IMAGE ANALYSIS
(2024)
Article
Computer Science, Artificial Intelligence
Riddhish Bhalodia, Shireen Elhabian, Jadie Adams, Wenzheng Tao, Ladislav Kavan, Ross Whitaker
Summary: This paper introduces DeepSSM, a deep learning-based framework for image-to-shape modeling. By learning the functional mapping from images to low-dimensional shape descriptors, DeepSSM can directly infer statistical representation of anatomy from 3D images. Compared to traditional methods, DeepSSM eliminates the need for heavy manual preprocessing and segmentation, and significantly improves computational time.
MEDICAL IMAGE ANALYSIS
(2024)
Article
Computer Science, Artificial Intelligence
Florentin Liebmann, Marco von Atzigen, Dominik Stutz, Julian Wolf, Lukas Zingg, Daniel Suter, Nicola A. Cavalcanti, Laura Leoty, Hooman Esfandiari, Jess G. Snedeker, Martin R. Oswald, Marc Pollefeys, Mazda Farshad, Philipp Furnstahl
Summary: This study presents a marker-less approach for automatic registration and real-time navigation of lumbar spinal fusion surgery using a deep neural network, avoiding radiation exposure and surgical errors. The method was validated on an ex-vivo surgery and a public dataset.
MEDICAL IMAGE ANALYSIS
(2024)
Article
Computer Science, Artificial Intelligence
Piyush Tiwary, Kinjawl Bhattacharyya, A. P. Prathosh
Summary: Domain shift refers to the change of distributional characteristics between training and testing datasets, leading to performance drop. For medical image tasks, domain shift can be caused by changes in imaging modalities, devices, and staining mechanisms. Existing approaches based on generative models suffer from training difficulties and lack of diversity. In this paper, the authors propose the use of energy-based models (EBMs) for unpaired image-to-image translation in medical images. The proposed method, called Cycle Consistent Twin EBMs (CCT-EBM), employs a pair of EBMs in the latent space of an Auto-Encoder to ensure translation symmetry and coupling between domains.
MEDICAL IMAGE ANALYSIS
(2024)
Article
Computer Science, Artificial Intelligence
Yutong Xie, Jianpeng Zhang, Lingqiao Liu, Hu Wang, Yiwen Ye, Johan Verjans, Yong Xia
Summary: This paper proposes a hybrid pre-training paradigm that combines self-supervised learning and supervised learning to improve the representation quality for medical image segmentation tasks. It introduces a reference task in self-supervised learning and optimizes the model using a gradient matching method. The experimental results demonstrate the effectiveness of this approach on multiple medical image segmentation benchmarks.
MEDICAL IMAGE ANALYSIS
(2024)
Article
Computer Science, Artificial Intelligence
Youyi Song, Jing Zou, Kup-Sze Choi, Baiying Lei, Jing Qin
Summary: Cell classification is crucial for intelligent cervical cancer screening, but the variation in cells' appearance and shape poses challenges. A new learning algorithm, worse-case boosting, is proposed to improve classification accuracy for under-represented data. Experimental results demonstrate the effectiveness of this algorithm in two publicly available datasets, achieving a 4% improvement in accuracy.
MEDICAL IMAGE ANALYSIS
(2024)
Article
Computer Science, Artificial Intelligence
Sangjoon Park, Eun Sun Lee, Kyung Sook Shin, Jeong Eun Lee, Jong Chul Ye
Summary: The increasing demand for AI systems to monitor human errors and abnormalities in healthcare presents challenges. This study presents a model called Medical X-VL, which is tailored for the medical domain and outperformed current state-of-the-art models in two medical image datasets. The model enables various zero-shot tasks for monitoring AI in the medical domain.
MEDICAL IMAGE ANALYSIS
(2024)
Article
Computer Science, Artificial Intelligence
Anna Klimovskaia Susmelj, Berkan Lafci, Firat Ozdemir, Neda Davoudi, Xose Luis Dean-Ben, Fernando Perez-Cruz, Daniel Razansky
Summary: Optoacoustic imaging is a technique that uses optical excitation and ultrasound detection for biological tissue imaging. The quality of the images depends on the extent of tomographic coverage provided by the ultrasound detector arrays. However, full coverage is not always possible due to experimental constraints. The proposed signal domain adaptation network aims to reduce limited-view artifacts in the images.
MEDICAL IMAGE ANALYSIS
(2024)
Article
Computer Science, Artificial Intelligence
Srijay Deshpande, Muhammad Dawood, Fayyaz Minhas, Nasir Rajpoot
Summary: In this work, a novel framework called SynCLay is proposed for automated synthesis of histology images based on user-defined cellular layouts. The framework can generate realistic and high-quality histology images with different cellular arrangements, which is helpful for studying the role of cells in the tumor microenvironment. The framework integrates a nuclear segmentation and classification model to refine nuclear structures and generate nuclear masks. Evaluation using quantitative metrics and feedback from pathologists shows that the synthetic images generated by SynCLay have high realism scores and can accurately differentiate between benign and malignant tumors.
MEDICAL IMAGE ANALYSIS
(2024)
Article
Computer Science, Artificial Intelligence
Ahmed H. Shahin, An Zhao, Alexander C. Whitehead, Daniel C. Alexander, Joseph Jacob, David Barber
Summary: Survival analysis is a valuable tool in healthcare for predicting the time to specific events. This paper introduces CenTime, a novel approach that directly estimates the time to event. The method performs well with censored data and can be easily integrated with deep learning models. Compared to standard methods, CenTime offers superior performance in predicting event time while maintaining comparable ranking performance.
MEDICAL IMAGE ANALYSIS
(2024)
Article
Computer Science, Artificial Intelligence
Bingyuan Liu, Jose Dolz, Adrian Galdran, Riadh Kobbi, Ismail Ben Ayed
Summary: Most segmentation losses, such as CE and Dice, are variants of the Cross-Entropy or Dice losses. This work provides a theoretical analysis that shows a deeper connection between CE and Dice than previously thought. From a constrained-optimization perspective, both CE and Dice decompose into similar ground-truth matching terms and region-size penalty terms. The analysis uncovers hidden region-size biases: Dice has an intrinsic bias towards extremely imbalanced solutions, while CE implicitly encourages the ground-truth region proportions. Based on this analysis, a principled and simple solution is proposed to explicitly control the region-size bias.
MEDICAL IMAGE ANALYSIS
(2024)
