4.6 Article

A fast hybrid retargeting scheme with seam context and content aware strip partition

Journal

NEUROCOMPUTING
Volume 286, Issue -, Pages 198-213

Publisher

ELSEVIER SCIENCE BV
DOI: 10.1016/j.neucom.2018.01.058

Keywords

Non-continuous seam carving; Seam context; Content aware strip partition; Seam distribution; Fast content aware image distance

Funding

  1. Beijing Municipal Education Commission Science and Technology Innovation Project [KZ201610005012]
  2. National Natural Science Foundation of China [61702022]
  3. China Postdoctoral Science Foundation [2017M610026]
  4. Beijing Postdoctoral Research Foundation [2017-ZZ-032]

Ask authors/readers for more resources

Image retargeting as a basic trick has been successfully applied to various computer vision problems. In this work, we propose a fast non-continuous seam carving with seam context (FN2SC) for robust and efficient image retargeting. FN2SC first conducts content aware image partition to separate an image into several strips with different tar get sizes from their content importance. It helps the removed seams in key objects distribute in a relatively uniform manner and prevents distortions to key objects. Then FN2SC performs fast non-continuous seam carving controlled by seam context of both neighboring relationship and touch bound relationship of seams. The seam context makes the removed seams distribute scattered and relieves artifacts caused by seam carving. During seam carving, image distortion is monitored by fast content aware image distance. Finally, FN2SC switches seam carving to scaling when the distortion meets the tolerance, which resizes the strips to the target sizes for image retargeting. Specifically, fast seam searching and image distortion based switching make FN2SC a fast and effective hybrid scheme. Experimental results demonstrate that the proposed FN2SC approach achieves good performance in terms of image quality and efficiency comprehensively. (c) 2018 Elsevier B.V. All rights reserved.

Authors

I am an author on this paper
Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

4.6
Not enough ratings

Secondary Ratings

Novelty
-
Significance
-
Scientific rigor
-
Rate this paper

Recommended

Article Computer Science, Artificial Intelligence

Multimodal collaborative graph for image recommendation

Meng Jian, Jingjing Guo, Ge Shi, Lifang Wu, Zhangquan Wang

Summary: This study presents a multimodal collaborative graph model for image recommendation, which uncovers users' interests by considering both visual and collaborative signals. Experimental results demonstrate the effectiveness of the model in mining users' interests and image recommendation.

APPLIED INTELLIGENCE (2023)

Article Computer Science, Software Engineering

Balance-Aware Grid Collage for Small Image Collections

Yu Song, Fan Tang, Weiming Dong, Feiyue Huang, Tong-Yee Lee, Changsheng Xu

Summary: This article focuses on the influence of visual effects on individual preferences in grid collages of small image collections. A novel balance-aware metric is proposed to evaluate the visual balance in line with human subjective perception. The metric integrates psychological achievements and incorporates a bonus mechanism for user preference. Experiments show that the metric can accurately evaluate grid collages and generate visually pleasant results comparable to manual designs.

IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS (2023)

Article Computer Science, Artificial Intelligence

Exploring the Temporal Consistency of Arbitrary Style Transfer: A Channelwise Perspective

Xiaoyu Kong, Yingying Deng, Fan Tang, Weiming Dong, Chongyang Ma, Yongyong Chen, Zhenyu He, Changsheng Xu

Summary: This article conducts a detailed analysis of the flickering effects in video stylization and proposes a new multichannel correlation network (MCCNet) to address this issue. MCCNet aligns each output frame with the input frame in the hidden feature space while preserving the desired style patterns.

IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS (2023)

Article Computer Science, Software Engineering

Towards harmonized regional style transfer and manipulation for facial images

Cong Wang, Fan Tang, Yong Zhang, Tieru Wu, Weiming Dong

Summary: This paper addresses the inconsistency issue of different regions in facial image synthesis and proposes a method for harmonized regional style transfer using a multi-scale encoder and a multi-region style attention module. By extracting style information from multiple regions of a reference image, a harmonious result is generated. The paper also introduces style mapping networks for multi-modal style synthesis and utilizes an invertible flow model to fine-tune the style code. Experimental results demonstrate that the proposed model can reliably perform style transfer and multi-modal manipulation, producing outputs comparable to state-of-the-art methods.

COMPUTATIONAL VISUAL MEDIA (2023)

Article Computer Science, Software Engineering

A Unified Arbitrary Style Transfer Framework via Adaptive Contrastive Learning

Yuxin Zhang, Fan Tang, Weiming Dong, Haibin Huang, Chongyang Ma, Tong-Yee Lee, Changsheng Xu

Summary: This work presents a novel framework, UCAST, for unified and arbitrary style transfer. By learning style representation and conducting contrastive learning, it addresses the issue of insufficient style information utilization in existing methods. The results obtained through contrastive learning and adaptive temperature scheme outperform state-of-the-art methods.

ACM TRANSACTIONS ON GRAPHICS (2023)

Article Engineering, Electrical & Electronic

Active Spatial Positions Based Hierarchical Relation Inference for Group Activity Recognition

Lifang Wu, Xianglong Lang, Ye Xiang, Changwen Chen, Zun Li, Zhuming Wang

Summary: This paper proposes a novel end-to-end hierarchical relation inference framework based on active spatial positions for group activity recognition. The framework is able to locate active spatial positions and use them as visual tokens to infer the relations for token embeddings. Experimental results show that the proposed framework only requires individual bounding box labeling in the training stage and automatically eliminates background noise from the entire scene.

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY (2023)

Article Computer Science, Information Systems

PhaseAnti: An Anti-Interference WiFi-Based Activity Recognition System Using Interference-Independent Phase Component

Jinyang Huang, Bin Liu, Chenglin Miao, Yan Lu, Qijia Zheng, Yu Wu, Jiancun Liu, Lu Su, Chang Wen Chen

Summary: Driven by various applications, recent achievements have been made in WiFi-based Human Activity Recognition (HAR) techniques that utilize WiFi infrastructures to infer human activities. However, the ubiquitous Co-channel Interference (CCI) poses a challenge to the performance of these systems. In this paper, a novel approach called PhaseAnti is proposed to exploit the CCI-independent phase component of WiFi Channel State Information (CSI) for HAR, which achieves superior effectiveness and recognition speed compared with existing solutions.

IEEE TRANSACTIONS ON MOBILE COMPUTING (2023)

Article Computer Science, Artificial Intelligence

SGCL: Spatial guided contrastive learning on whole-slide pathological images

Tiancheng Lin, Zhimiao Yu, Zengchao Xu, Hongyu Hu, Yi Xu, Chang-Wen Chen

Summary: Self-supervised representation learning has limitations when applied to whole-slide pathological images due to their unique characteristics. To address this issue, we propose a novel scheme called Spatial Guided Contrastive Learning (SGCL), which leverages spatial proximity and multi-object priors for stable self-supervision. SGCL outperforms state-of-the-art methods on diverse downstream tasks across multiple datasets.

MEDICAL IMAGE ANALYSIS (2023)

Article Computer Science, Artificial Intelligence

CrossRectify: Leveraging disagreement for semi-supervised object detection

Chengcheng Ma, Xingjia Pan, Qixiang Ye, Fan Tang, Weiming Dong, Changsheng Xu

Summary: Semi-supervised object detection has made significant progress, but the performance of self-labeling-based methods is limited due to the misguidance of incorrect pseudo labels predicted by the detector itself. This paper proposes a detection framework called CrossRectify, which obtains accurate pseudo labels by simultaneously training two detectors with different initial parameters. The proposed approach leverages the disagreements between detectors to discern self-errors and improves the quality of pseudo labels. Extensive experiments demonstrate that CrossRectify achieves superior performance on 2D and 3D detection benchmarks.

PATTERN RECOGNITION (2023)

Article Engineering, Electrical & Electronic

Understanding and Mitigating Overfitting in Prompt Tuning for Vision-Language Models

Chengcheng Ma, Yang Liu, Jiankang Deng, Lingxi Xie, Weiming Dong, Changsheng Xu

Summary: Pretrained vision-language models like CLIP have strong generalization capability. The Context Optimization (CoOp) method has been proposed to improve performance by learning continuous prompts. However, CoOp suffers from overfitting. This study analyzes the gradient flow and proposes Subspace Prompt Tuning (Sub PT) to eliminate overfitting. Additionally, a Novel Feature Learner (NFL) is introduced to enhance CoOp's generalization ability.

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY (2023)

Article Computer Science, Artificial Intelligence

Reciprocal normalization for domain adaptation

Zhiyong Huang, Kekai Sheng, Ke Li, Jian Liang, Taiping Yao, Weiming Dong, Dengwen Zhou, Xing Sun

Summary: Batch normalization is widely used in deep neural networks, but it is ineffective for cross-domain tasks. This paper proposes a novel normalization method called Reciprocal Normalization (RN), which utilizes cross-domain relation to improve adaptability. Compared to batch normalization, RN is more suitable for unsupervised domain adaptation and can be easily integrated into popular domain adaptation methods.

PATTERN RECOGNITION (2023)

Article Art

Portrait Map Art Generation By Asymmetric Image-to-Image Translation

Yuxin Zhang, Fan Tang, Weiming Dong, Thi-Ngoc-Hanh Le, Changsheng Xu, Tong-Yee Lee

Summary: The authors propose a deep neural network-based algorithm to automatically generate Portrait Map Art (PMA), a modern art form created by British portrait artist Ed Fairburn. They formulate the generation of PMA as an adaptive dual-to-single image translation problem and utilize encoder networks to analyze the appearance of the portrait and map images. The proposed model can produce new PMA images without baselines by optimizing with a cycle-consistency constraint.

LEONARDO (2023)

Article Computer Science, Information Systems

User-Guided Personalized Image Aesthetic Assessment Based on Deep Reinforcement Learning

Pei Lv, Jianqi Fan, Xixi Nie, Weiming Dong, Xiaoheng Jiang, Bing Zhou, Mingliang Xu, Changsheng Xu

Summary: Personalized image aesthetic assessment is a hot topic with wide applications in various fields. We propose a novel user-guided framework based on deep reinforcement learning to acquire precise personalized aesthetic distribution by small amount of samples. Our approach achieves state-of-the-art results in personalized image aesthetic assessment tasks.

IEEE TRANSACTIONS ON MULTIMEDIA (2023)

Article Computer Science, Artificial Intelligence

SPA2Net: Structure-Preserved Attention Activated Network for Weakly Supervised Object Localization

Dong Chen, Xingjia Pan, Fan Tang, Weiming Dong, Changsheng Xu

Summary: In this paper, a new weakly supervised object localization framework called SPA(2)Net is proposed to accurately locate objects by preserving the structure and utilizing spatial attention in deep CNN. Unlike traditional methods, SPA(2)Net separates the localization task from the classification task by introducing a localization branch and optimizing it with a self-supervised structural-preserved localization mask. Experimental results show that SPA(2)Net achieves substantial and consistent performance gains compared to baseline approaches on two benchmark datasets.

IEEE TRANSACTIONS ON IMAGE PROCESSING (2023)

Proceedings Paper Computer Science, Artificial Intelligence

Graph Contrastive Learning on Complementary Embedding for Recommendation

Meishan Liu, Meng Jian, Ge Shi, Ye Xiang, Lifang Wu

Summary: In this paper, we propose a graph contrastive learning on complementary embedding (GCCE) approach to address the challenges of interaction sparsity and bias by introducing negative interests and designing perturbed graph convolution and complementary embedding generation for interest modeling. We validate the effectiveness of GCCE through experiments on real datasets and show its superiority over state-of-the-art recommendation models.

PROCEEDINGS OF THE 2023 ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA RETRIEVAL, ICMR 2023 (2023)

Article Computer Science, Artificial Intelligence

3D-KCPNet: Efficient 3DCNNs based on tensor mapping theory

Rui Lv, Dingheng Wang, Jiangbin Zheng, Zhao-Xu Yang

Summary: In this paper, the authors investigate tensor decomposition for neural network compression. They analyze the convergence and precision of tensor mapping theory, validate the rationality of tensor mapping and its superiority over traditional tensor approximation based on the Lottery Ticket Hypothesis. They propose an efficient method called 3D-KCPNet to compress 3D convolutional neural networks using the Kronecker canonical polyadic (KCP) tensor decomposition. Experimental results show that 3D-KCPNet achieves higher accuracy compared to the original baseline model and the corresponding tensor approximation model.

NEUROCOMPUTING (2024)

Article Computer Science, Artificial Intelligence

Personalized robotic control via constrained multi-objective reinforcement learning

Xiangkun He, Zhongxu Hu, Haohan Yang, Chen Lv

Summary: In this paper, a novel constrained multi-objective reinforcement learning algorithm is proposed for personalized end-to-end robotic control with continuous actions. The approach trains a single model using constraint design and a comprehensive index to achieve optimal policies based on user-specified preferences.

NEUROCOMPUTING (2024)

Article Computer Science, Artificial Intelligence

Overlapping community detection using expansion with contraction

Zhijian Zhuo, Bilian Chen, Shenbao Yu, Langcai Cao

Summary: In this paper, a novel method called Expansion with Contraction Method for Overlapping Community Detection (ECOCD) is proposed, which utilizes non-negative matrix factorization to obtain disjoint communities and applies expansion and contraction processes to adjust the degree of overlap. ECOCD is applicable to various networks with different properties and achieves high-quality overlapping community detection.

NEUROCOMPUTING (2024)

Article Computer Science, Artificial Intelligence

High-compressed deepfake video detection with contrastive spatiotemporal distillation

Yizhe Zhu, Chunhui Zhang, Jialin Gao, Xin Sun, Zihan Rui, Xi Zhou

Summary: In this work, the authors propose a Contrastive Spatio-Temporal Distilling (CSTD) approach to improve the detection of high-compressed deepfake videos. The approach leverages spatial-frequency cues and temporal-contrastive alignment to fully exploit spatiotemporal inconsistency information.

NEUROCOMPUTING (2024)

Review Computer Science, Artificial Intelligence

A review of coverless steganography

Laijin Meng, Xinghao Jiang, Tanfeng Sun

Summary: This paper provides a review of coverless steganographic algorithms, including the development process, known contributions, and general issues in image and video algorithms. It also discusses the security of coverless steganography from theoretical analysis to actual investigation for the first time.

NEUROCOMPUTING (2024)

Article Computer Science, Artificial Intelligence

Confidence-based interactable neural-symbolic visual question answering

Yajie Bao, Tianwei Xing, Xun Chen

Summary: Visual question answering requires processing multi-modal information and effective reasoning. Neural-symbolic learning is a promising method, but current approaches lack uncertainty handling and can only provide a single answer. To address this, we propose a confidence based neural-symbolic approach that evaluates NN inferences and conducts reasoning based on confidence.

NEUROCOMPUTING (2024)

Article Computer Science, Artificial Intelligence

A framework-based transformer and knowledge distillation for interior style classification

Anh H. Vo, Bao T. Nguyen

Summary: Interior style classification is an interesting problem with potential applications in both commercial and academic domains. This project proposes a method named ISC-DeIT, which combines data-efficient image transformer architectures and knowledge distillation, to address the interior style classification problem. Experimental results demonstrate a significant improvement in predictive accuracy compared to other state-of-the-art methods.

NEUROCOMPUTING (2024)

Article Computer Science, Artificial Intelligence

Improving robustness for vision transformer with a simple dynamic scanning augmentation

Shashank Kotyan, Danilo Vasconcellos Vargas

Summary: This article introduces a novel augmentation technique called Dynamic Scanning Augmentation to improve the accuracy and robustness of Vision Transformer (ViT). The technique leverages dynamic input sequences to adaptively focus on different patches, resulting in significant changes in ViT's attention mechanism. Experimental results demonstrate that Dynamic Scanning Augmentation outperforms ViT in terms of both robustness to adversarial attacks and accuracy against natural images.

NEUROCOMPUTING (2024)

Article Computer Science, Artificial Intelligence

Introducing shape priors in Siamese networks for image classification

Hiba Alqasir, Damien Muselet, Christophe Ducottet

Summary: The article proposes a solution to improve the learning process of a classification network by providing shape priors, reducing the need for annotated data. The solution is tested on cross-domain digit classification tasks and a video surveillance application.

NEUROCOMPUTING (2024)

Article Computer Science, Artificial Intelligence

Neural dynamics solver for time-dependent infinity-norm optimization based on ACP framework with robot application

Dexiu Ma, Mei Liu, Mingsheng Shang

Summary: This paper proposes a method using neural dynamics solvers to solve infinity-norm optimization problems. Two improved solvers are constructed and their effectiveness and superiority are demonstrated through theoretical analysis and simulation experiments.

NEUROCOMPUTING (2024)

Article Computer Science, Artificial Intelligence

cpp-AIF: A multi-core C plus plus implementation of Active Inference for Partially Observable Markov Decision Processes

Francesco Gregoretti, Giovanni Pezzulo, Domenico Maisto

Summary: Active Inference is a computational framework that uses probabilistic inference and variational free energy minimization to describe perception, planning, and action. cpp-AIF is a header-only C++ library that provides a powerful tool for implementing Active Inference for Partially Observable Markov Decision Processes through multi-core computing. It is cross-platform and improves performance, memory management, and usability compared to existing software.

NEUROCOMPUTING (2024)

Article Computer Science, Artificial Intelligence

Predicting stock market trends with self-supervised learning

Zelin Ying, Dawei Cheng, Cen Chen, Xiang Li, Peng Zhu, Yifeng Luo, Yuqi Liang

Summary: This paper proposes a novel stock market trends prediction framework called SMART, which includes a self-supervised stock technical data sequence embedding model S3E. By training with multiple self-supervised auxiliary tasks, the model encodes stock technical data sequences into embeddings and uses the learned sequence embeddings for predicting stock market trends. Extensive experiments on China A-Shares market and NASDAQ market prove the high effectiveness of our model in stock market trends prediction, and its effectiveness is further validated in real-world applications in a leading financial service provider in China.

NEUROCOMPUTING (2024)

Article Computer Science, Artificial Intelligence

DHGAT: Hyperbolic representation learning on dynamic graphs via attention networks

Hao Li, Hao Jiang, Dongsheng Ye, Qiang Wang, Liang Du, Yuanyuan Zeng, Liu Yuan, Yingxue Wang, C. Chen

Summary: DHGAT1, a dynamic hyperbolic graph attention network, utilizes hyperbolic metric properties to embed dynamic graphs. It employs a spatiotemporal self-attention mechanism and weighted node representations, resulting in excellent performance in link prediction tasks.

NEUROCOMPUTING (2024)

Article Computer Science, Artificial Intelligence

Progressive network based on detail scaling and texture extraction: A more general framework for image deraining

Jiehui Huang, Zhenchao Tang, Xuedong He, Jun Zhou, Defeng Zhou, Calvin Yu-Chian Chen

Summary: This study proposes a progressive learning multi-scale feature blending model for image deraining tasks. The model utilizes detail dilation and texture extraction to improve the restoration of rainy images. Experimental results show that the model achieves near state-of-the-art performance in rain removal tasks and exhibits better rain removal realism.

NEUROCOMPUTING (2024)

Article Computer Science, Artificial Intelligence

Stabilization and synchronization control for discrete-time complex networks via the auxiliary role of edges subsystem

Lizhi Liu, Zilin Gao, Yinhe Wang, Yongfu Li

Summary: This paper proposes a novel discrete-time interconnected model for depicting complex dynamical networks. The model consists of nodes and edges subsystems, which consider the dynamic characteristic of both nodes and edges. By designing control strategies and coupling modes, the stabilization and synchronization of the network are achieved. Simulation results demonstrate the effectiveness of the proposed methods.

NEUROCOMPUTING (2024)