Article
Engineering, Electrical & Electronic
Rong-Cheng Tu, Jie Jiang, Qinghong Lin, Chengfei Cai, Shangxuan Tian, Hongfa Wang, Wei Liu
Summary: In this paper, the authors propose a novel unsupervised cross-modal hashing method (UCHM) that utilizes a modality-interaction-enabled similarity generator and a bit-selection module to improve the retrieval performance of unlabeled cross-modal data.
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY
(2023)
Article
Computer Science, Information Systems
Rintaro Yanagi, Ren Togo, Takahiro Ogawa, Miki Haseyama
Summary: In this study, a novel interactive cross-modal image-retrieval method based on question answering is proposed. The method analyzes candidate images and asks users questions to obtain information, leading to the retrieval of desired images even with ambiguous query texts.
ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS
(2022)
Article
Mathematics
Xiaohan Yang, Zhen Wang, Nannan Wu, Guokun Li, Chuang Feng, Pingping Liu
Summary: This paper introduces the task of image-text cross-modal retrieval and the proposed DRNPH method, which achieves cross-modal retrieval in the Hamming space, with constraints for consistent binary codes of similar sample pairs and minimal Hamming distances. Experimental results show that this method outperforms existing methods in various image-text retrieval scenarios.
Article
Computer Science, Artificial Intelligence
Xiaohan Yang, Zhen Wang, Wenhao Liu, Xinyi Chang, Nannan Wu
Summary: In recent years, researchers have been using hashing algorithms to improve the efficiency of large-scale cross-modal retrieval by mapping features into binary codes. However, existing cross-modal hashing algorithms often overlook the multi-label information by focusing only on single class labels. To address this issue, we propose DAMCH, a deep adversarial multi-label cross-modal hashing algorithm that considers both multi-label and deep features. Our algorithm preserves the Hamming neighbor relationship and ensures the same semantic information in binary features as in the original label. Additionally, our algorithm minimizes information loss during feature mapping and ensures consistent feature distribution across modalities. Experimental results show that DAMCH outperforms state-of-the-art methods.
INTERNATIONAL JOURNAL OF MULTIMEDIA INFORMATION RETRIEVAL
(2023)
Article
Chemistry, Multidisciplinary
Wenxiao Li, Hongyan Mei, Yutian Li, Jiayao Yu, Xing Zhang, Xiaorong Xue, Jiahao Wang
Summary: This paper proposes Tri-CMH, a cross-modal hash retrieval method with fused triples, which effectively utilizes the dataset's supervisory information and improves the model's ability to judge cross-modality semantic similarity.
APPLIED SCIENCES-BASEL
(2023)
Article
Computer Science, Artificial Intelligence
Zhan Yang, Xiyin Deng, Jun Long
Summary: Hashing is an effective technique for large-scale data storage and efficient retrieval, and it plays a crucial role in the intelligent development of new infrastructure. Unsupervised cross-modal hashing techniques have gained extensive attention due to their fast retrieval speed and feasibility. However, existing methods are insufficient in describing the complex relations among different modalities, such as the balance between complementarity and consistency.
NEURAL COMPUTING & APPLICATIONS
(2023)
Article
Computer Science, Artificial Intelligence
Zhong Zhang, Qing Dong, Sen Wang, Shuang Liu, Baihua Xiao, Tariq S. Durrani
Summary: This paper proposes a hybrid mutual learning method for cross-modality person re-identification, aiming to establish a collaborative relationship between RGB modality and IR modality. The method reduces the distribution gap by mutual learning from local features and triplet relations, and fuses feature information using hierarchical attention aggregation.
IET COMPUTER VISION
(2023)
Article
Computer Science, Information Systems
Chengyuan Zhang, Jiayu Song, Xiaofeng Zhu, Lei Zhu, Shichao Zhang
Summary: The Hybrid Cross-Modal Similarity Learning model (HCMSL) proposed in this article effectively addresses the similarity measurement issue in cross-modal retrieval by capturing semantic information and establishing a common subspace between different modalities. Comprehensive experiments demonstrate significant improvements over existing techniques on real datasets.
ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS
(2021)
Article
Computer Science, Artificial Intelligence
Asad Khan, Sakander Hayat, Muhammad Ahmad, Jinyu Wen, Muhammad Umar Farooq, Meie Fang, Wenchao Jiang
Summary: Cross-modal retrieval has gained great attention recently due to the increasing demand for multimodal data. This research proposes two cross-modal recovery techniques based on a dual-branch neural network and hashing learning method, which effectively address the problem of inappropriate information between images and texts by establishing a common subspace.
INTERNATIONAL JOURNAL OF INTELLIGENT SYSTEMS
(2022)
Article
Computer Science, Information Systems
Qiubin Lin, Wenming Cao, Zhiquan He, Zhihai He
Summary: The rapid development of deep learning has led to significant progress in cross-modal retrieval and the recent attention towards cross-modal hashing. The existing semantic heterogeneity gap between different modalities presents a challenging problem. To address this, we propose the MDCH approach, which introduces semantic mask information and alternately trains intra-modal and inter-modal networks to improve hash code effectiveness.
IEEE TRANSACTIONS ON MULTIMEDIA
(2021)
Article
Computer Science, Artificial Intelligence
Gaurav Patel, Jose Dolz
Summary: This paper presents a novel learning strategy that leverages self-supervision to significantly enhance class activation maps (CAMs) in a multi-modal image scenario. The proposed method effectively improves the performance of the model through the introduction of equivariant terms in the loss function and KL-divergence, and outperforms relevant literature on multiple datasets.
MEDICAL IMAGE ANALYSIS
(2022)
Article
Computer Science, Information Systems
Yicai Xie, Xianhua Zeng, Tinghua Wang, Yun Yi
Summary: A novel method called ODHUC is proposed for both uni-modal and cross-modal retrieval. It adopts an online deep hashing approach to continuously learn hash codes by sampling and updating the model. ODHUC also avoids forgetting old knowledge through knowledge distillation. Experimental results demonstrate that ODHUC outperforms other methods.
INFORMATION SCIENCES
(2022)
Article
Engineering, Electrical & Electronic
Yunbo Wang, Yuxin Peng
Summary: This paper proposes a cross-media retrieval method called MARS, which allows each modality to be trained independently, improving the flexibility and practicality of CMR. MARS introduces a label parsing module and a modality-specific representation module to generate modality-agnostic semantic representation and trains them using the same objective for better semantic alignment. Experimental results demonstrate that MARS outperforms existing methods.
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY
(2022)
Article
Computer Science, Information Systems
Qingzhen Xu, Shuang Liu, Han Qiao, Miao Li
Summary: This paper proposes a dual optimization method (CMRDO) for cross-modal retrieval, which improves retrieval accuracy by optimizing the common representation space and introducing an efficient sample construction strategy, and has strong generalization ability.
MULTIMEDIA TOOLS AND APPLICATIONS
(2023)
Article
Mathematics, Interdisciplinary Applications
Hui Li
Summary: This study implements a tourism network resource monitoring system with topic collection and extraction algorithms, successfully extracting main content from a variety of web pages to promote the construction of tourism informatization.
Article
Computer Science, Software Engineering
Shouqiang Liu, Miao Li, Min Li, Qingzhen Xu
CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE
(2020)
Article
Computer Science, Information Systems
Qingzhen Xu, Shuang Liu, Han Qiao, Miao Li
Summary: This paper proposes a dual optimization method (CMRDO) for cross-modal retrieval, which improves retrieval accuracy by optimizing the common representation space and introducing an efficient sample construction strategy, and has strong generalization ability.
MULTIMEDIA TOOLS AND APPLICATIONS
(2023)
Article
Computer Science, Information Systems
Guangyi Huang, Yongyi Gong, Qingzhen Xu, Kanoksak Wattanachote, Kun Zeng, Xiaonan Luo