Journal
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY
Volume 31, Issue 12, Pages 4798-4811Publisher
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TCSVT.2021.3055197
Keywords
Task analysis; Feature extraction; Distortion; Image quality; Databases; Visualization; Semantics; Image quality assessment; multi-task learning; saliency feature fusion; convolutional neural network
Categories
Funding
- National Natural Science Foundation of China [62071369]
- Key Research and Development Program of Shaanxi Province [2020KW-009]
Ask authors/readers for more resources
This study proposes an end-to-end multi-task deep convolution neural network that jointly optimizes IQ and saliency subtasks to improve saliency-guided IQ performance. By progressively improving IQ features over network depth and successfully incorporating saliency information, the network achieves state-of-the-art performance and strong generalization ability on IQ databases.
As the evaluation of image quality depends on the human visual system (HVS), many existing image quality assessment (IQA) methods focus on modeling the HVS to account for subjective perception. The visual attention of the HVS makes humans more sensitive to distortion on the attended regions than on regions which are not the focus of attention. Therefore, we propose an end-to-end multi-task deep convolution neural network with multi-scale and multi-hierarchy fusion (MMMNet), in which the IQA and saliency subtasks are jointly optimized to improve saliency-guided IQA performance. Particularly, the incorporation of saliency information is achieved by fusing saliency features with IQA features hierarchically to progressively improve the IQA features over network depth. A multi-scale feature extraction module (MSFE) is proposed to provide effective saliency features for the IQA network. Based on the saliency fusion, MMMNet introduces an auxiliary saliency task, achieving the multi-task learning to improve the generalization of the IQA task. Experimental results show that MMMNet achieves state-of-the-art performance and strong generalization ability on IQA databases.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available