Perception-guided multi-channel visual feature fusion for image retargeting

Image retargeting aims to arbitrarily change the aspect ratio of images with minimal visual artifacts, that is, preserving salient regions within the image. Conventional approaches have achieved impressive performance, however, only low-level features are exploited. Besides, these approaches do not take human visual system into account. To solve these problems, we propose a perception-guided multi-channel visual feature fusion method for image retargeting, where human visual perception is well encoded. Specifically, we first construct a series of graphlets which represent salient regions that might attract human attention, where both low-level and high-level features are utilized to describe graphlets. Considering that only a few regions can attract human attention when observing the image, we engineer a sparsity-constraint algorithm to select these regions, which will be further concatenated to form human gaze shifting paths (GSPs). Subsequently, we design a statistic based CNN architecture to extract the deep representation of GSPs. Afterward, a probabilistic model is built to learn the priors of GSPs and the learned probabilistic model will guide image retargeting. A test image whose distribution is similar to the learned probabilistic model is deemed as the aesthetic image and thus should be shrunk slightly. Comprehensive experiments demonstrate the effectiveness and robustness of the proposed method.